* futex priority based wakeup
@ 2007-09-07 15:02 Benedict, Michael
2007-09-07 16:41 ` Ilya Lipovsky
0 siblings, 1 reply; 12+ messages in thread
From: Benedict, Michael @ 2007-09-07 15:02 UTC (permalink / raw)
To: linuxppc-embedded
I recently upgraded to 2.6.22 to get the priority based futex wakeup
behavior. However, when I run a simple program (see below), based on
either a pthread_mutex_t or a sem_t, it seems that threads are woken up
in FIFO order. I am using glibc 2.3.6 with NPTL and TLS, based off
crossdev-0.43. Could someone help me get priority-based wakeup or point
me to a better place to get this question answered?
Thank you,
Michael
Code:
pthread_mutex_t mymutex =3D PTHREAD_MUTEX_INITIALIZER;
void *important(void *ign)
{
sleep(1);
printf("important waiting for mutex\n");
if(pthread_mutex_lock(&mymutex)) {
perror("sem_wait");
exit(1);
} else {
printf("important got mutex!\n");
pthread_mutex_unlock(&mymutex);
}
return NULL;
}
void *unimportant(void *ign)
{
printf("unimportant waiting for mutex\n");
if(pthread_mutex_lock(&mymutex)) {
perror("sem_wait");
exit(1);
} else {
printf("unimportant got mutex!\n");
pthread_mutex_unlock(&mymutex);
}
return NULL;
}
int main()
{
struct sched_param p;
pthread_attr_t attr;
pthread_t i, u;
pthread_mutex_lock(&mymutex);
p.__sched_priority =3D sched_get_priority_min(SCHED_FIFO);
if(-1 =3D=3D p.__sched_priority) {
perror("sched_get_priority_min");
return 1;
}
pthread_attr_init(&attr);
pthread_attr_setschedpolicy(&attr, SCHED_FIFO);
pthread_attr_setschedparam(&attr, &p);
pthread_create(&u, &attr, unimportant, NULL);
p.__sched_priority =3D sched_get_priority_max(SCHED_FIFO);
pthread_attr_setschedparam(&attr, &p);
pthread_create(&i, &attr, important, NULL);
sleep(5);
printf("main unlocking mutex\n");
pthread_mutex_unlock(&mymutex);
pthread_join(u, NULL);
pthread_join(i, NULL);
return 0;
}
Which produces:
unimportant waiting for mutex
important waiting for mutex
main unlocking mutex
unimportant got mutex!
important got mutex!
^ permalink raw reply [flat|nested] 12+ messages in thread* RE: futex priority based wakeup 2007-09-07 15:02 futex priority based wakeup Benedict, Michael @ 2007-09-07 16:41 ` Ilya Lipovsky 2007-09-07 17:16 ` Ilya Lipovsky 0 siblings, 1 reply; 12+ messages in thread From: Ilya Lipovsky @ 2007-09-07 16:41 UTC (permalink / raw) To: 'Benedict, Michael', linuxppc-embedded Looks like it is a libc issue. Apparently, GNU libc supports priority futexes only in version 2.5 and higher. -----Original Message----- From: linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org [mailto:linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org] On Behalf Of Benedict, Michael Sent: Friday, September 07, 2007 11:02 AM To: linuxppc-embedded@ozlabs.org Subject: futex priority based wakeup I recently upgraded to 2.6.22 to get the priority based futex wakeup behavior. However, when I run a simple program (see below), based on either a pthread_mutex_t or a sem_t, it seems that threads are woken up in FIFO order. I am using glibc 2.3.6 with NPTL and TLS, based off crossdev-0.43. Could someone help me get priority-based wakeup or point me to a better place to get this question answered? Thank you, Michael Code: pthread_mutex_t mymutex = PTHREAD_MUTEX_INITIALIZER; void *important(void *ign) { sleep(1); printf("important waiting for mutex\n"); if(pthread_mutex_lock(&mymutex)) { perror("sem_wait"); exit(1); } else { printf("important got mutex!\n"); pthread_mutex_unlock(&mymutex); } return NULL; } void *unimportant(void *ign) { printf("unimportant waiting for mutex\n"); if(pthread_mutex_lock(&mymutex)) { perror("sem_wait"); exit(1); } else { printf("unimportant got mutex!\n"); pthread_mutex_unlock(&mymutex); } return NULL; } int main() { struct sched_param p; pthread_attr_t attr; pthread_t i, u; pthread_mutex_lock(&mymutex); p.__sched_priority = sched_get_priority_min(SCHED_FIFO); if(-1 == p.__sched_priority) { perror("sched_get_priority_min"); return 1; } pthread_attr_init(&attr); pthread_attr_setschedpolicy(&attr, SCHED_FIFO); pthread_attr_setschedparam(&attr, &p); pthread_create(&u, &attr, unimportant, NULL); p.__sched_priority = sched_get_priority_max(SCHED_FIFO); pthread_attr_setschedparam(&attr, &p); pthread_create(&i, &attr, important, NULL); sleep(5); printf("main unlocking mutex\n"); pthread_mutex_unlock(&mymutex); pthread_join(u, NULL); pthread_join(i, NULL); return 0; } Which produces: unimportant waiting for mutex important waiting for mutex main unlocking mutex unimportant got mutex! important got mutex! _______________________________________________ Linuxppc-embedded mailing list Linuxppc-embedded@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-embedded ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: futex priority based wakeup 2007-09-07 16:41 ` Ilya Lipovsky @ 2007-09-07 17:16 ` Ilya Lipovsky 2007-09-07 17:24 ` Benedict, Michael 0 siblings, 1 reply; 12+ messages in thread From: Ilya Lipovsky @ 2007-09-07 17:16 UTC (permalink / raw) To: 'Benedict, Michael', linuxppc-embedded ...Or maybe I am wrong :). Could you put fflush(stdout) after each printf, just to be completely certain that it misbehaves? -----Original Message----- From: linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org [mailto:linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org] On Behalf Of Ilya Lipovsky Sent: Friday, September 07, 2007 12:41 PM To: 'Benedict, Michael'; linuxppc-embedded@ozlabs.org Subject: RE: futex priority based wakeup Looks like it is a libc issue. Apparently, GNU libc supports priority futexes only in version 2.5 and higher. -----Original Message----- From: linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org [mailto:linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org] On Behalf Of Benedict, Michael Sent: Friday, September 07, 2007 11:02 AM To: linuxppc-embedded@ozlabs.org Subject: futex priority based wakeup I recently upgraded to 2.6.22 to get the priority based futex wakeup behavior. However, when I run a simple program (see below), based on either a pthread_mutex_t or a sem_t, it seems that threads are woken up in FIFO order. I am using glibc 2.3.6 with NPTL and TLS, based off crossdev-0.43. Could someone help me get priority-based wakeup or point me to a better place to get this question answered? Thank you, Michael Code: pthread_mutex_t mymutex = PTHREAD_MUTEX_INITIALIZER; void *important(void *ign) { sleep(1); printf("important waiting for mutex\n"); if(pthread_mutex_lock(&mymutex)) { perror("sem_wait"); exit(1); } else { printf("important got mutex!\n"); pthread_mutex_unlock(&mymutex); } return NULL; } void *unimportant(void *ign) { printf("unimportant waiting for mutex\n"); if(pthread_mutex_lock(&mymutex)) { perror("sem_wait"); exit(1); } else { printf("unimportant got mutex!\n"); pthread_mutex_unlock(&mymutex); } return NULL; } int main() { struct sched_param p; pthread_attr_t attr; pthread_t i, u; pthread_mutex_lock(&mymutex); p.__sched_priority = sched_get_priority_min(SCHED_FIFO); if(-1 == p.__sched_priority) { perror("sched_get_priority_min"); return 1; } pthread_attr_init(&attr); pthread_attr_setschedpolicy(&attr, SCHED_FIFO); pthread_attr_setschedparam(&attr, &p); pthread_create(&u, &attr, unimportant, NULL); p.__sched_priority = sched_get_priority_max(SCHED_FIFO); pthread_attr_setschedparam(&attr, &p); pthread_create(&i, &attr, important, NULL); sleep(5); printf("main unlocking mutex\n"); pthread_mutex_unlock(&mymutex); pthread_join(u, NULL); pthread_join(i, NULL); return 0; } Which produces: unimportant waiting for mutex important waiting for mutex main unlocking mutex unimportant got mutex! important got mutex! _______________________________________________ Linuxppc-embedded mailing list Linuxppc-embedded@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-embedded _______________________________________________ Linuxppc-embedded mailing list Linuxppc-embedded@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-embedded ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: futex priority based wakeup 2007-09-07 17:16 ` Ilya Lipovsky @ 2007-09-07 17:24 ` Benedict, Michael 2007-09-07 17:45 ` Ilya Lipovsky 0 siblings, 1 reply; 12+ messages in thread From: Benedict, Michael @ 2007-09-07 17:24 UTC (permalink / raw) To: linuxppc-embedded Thank you for the help! Same result with fflush :(. Are you maybe wrong because you have tested 2.5 and it doesn't work? Or just uncertain? The only thing I have read that mentions glibc version is the work Ingo did against 2.4 glibc and in the 2.6.16-mm1 patches... http://people.redhat.com/mingo/PI-futex-patches/. I can not detrmine how this relates to the 2.6.22 priority based futex wakeup commit. I am currently fighting crosstool-0.43 to build a newer glibc and I will share results if/when I get that to work. -Michael Ilya Lipovsky wrote: > ...Or maybe I am wrong :). Could you put fflush(stdout) after > each printf, > just to be completely certain that it misbehaves? >=20 > -----Original Message----- > From: linuxppc-embedded-bounces+lipovsky=3Dcs.bu.edu@ozlabs.org > [mailto:linuxppc-embedded-bounces+lipovsky=3Dcs.bu.edu@ozlabs.or g] On > Behalf Of Ilya Lipovsky Sent: Friday, September 07, 2007 12:41 PM > To: 'Benedict, Michael'; linuxppc-embedded@ozlabs.org > Subject: RE: futex priority based wakeup >=20 > Looks like it is a libc issue. Apparently, GNU libc supports priority > futexes only in version 2.5 and higher. >=20 > -----Original Message----- > From: linuxppc-embedded-bounces+lipovsky=3Dcs.bu.edu@ozlabs.org > [mailto:linuxppc-embedded-bounces+lipovsky=3Dcs.bu.edu@ozlabs.or g] On > Behalf Of Benedict, Michael Sent: Friday, September 07, 2007 11:02 AM > To: linuxppc-embedded@ozlabs.org > Subject: futex priority based wakeup >=20 > I recently upgraded to 2.6.22 to get the priority based futex wakeup > behavior. However, when I run a simple program (see below), based on > either a pthread_mutex_t or a sem_t, it seems that threads > are woken up > in FIFO order. I am using glibc 2.3.6 with NPTL and TLS, based off > crossdev-0.43. Could someone help me get priority-based > wakeup or point > me to a better place to get this question answered? Thank you, > Michael >=20 > Code: >=20 > pthread_mutex_t mymutex =3D PTHREAD_MUTEX_INITIALIZER; >=20 > void *important(void *ign) > { > sleep(1); > printf("important waiting for mutex\n"); > if(pthread_mutex_lock(&mymutex)) { > perror("sem_wait"); > exit(1); > } else { > printf("important got mutex!\n"); > pthread_mutex_unlock(&mymutex); > } >=20 > return NULL; > } >=20 >=20 > void *unimportant(void *ign) > { > printf("unimportant waiting for mutex\n"); > if(pthread_mutex_lock(&mymutex)) { > perror("sem_wait"); > exit(1); > } else { > printf("unimportant got mutex!\n"); > pthread_mutex_unlock(&mymutex); > } >=20 > return NULL; > } >=20 > int main() > { > struct sched_param p; > pthread_attr_t attr; > pthread_t i, u; >=20 > pthread_mutex_lock(&mymutex); >=20 > p.__sched_priority =3D sched_get_priority_min(SCHED_FIFO); > if(-1 =3D=3D p.__sched_priority) { > perror("sched_get_priority_min"); > return 1; > } > pthread_attr_init(&attr); > pthread_attr_setschedpolicy(&attr, SCHED_FIFO); > pthread_attr_setschedparam(&attr, &p); > pthread_create(&u, &attr, unimportant, NULL); >=20 > p.__sched_priority =3D sched_get_priority_max(SCHED_FIFO); > pthread_attr_setschedparam(&attr, &p); > pthread_create(&i, &attr, important, NULL); >=20 > sleep(5); > printf("main unlocking mutex\n"); > pthread_mutex_unlock(&mymutex); >=20 > pthread_join(u, NULL); > pthread_join(i, NULL); >=20 > return 0; > } >=20 > Which produces: > unimportant waiting for mutex > important waiting for mutex > main unlocking mutex > unimportant got mutex! > important got mutex! >=20 > _______________________________________________ > Linuxppc-embedded mailing list > Linuxppc-embedded@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-embedded >=20 > _______________________________________________ > Linuxppc-embedded mailing list > Linuxppc-embedded@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-embedded ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: futex priority based wakeup 2007-09-07 17:24 ` Benedict, Michael @ 2007-09-07 17:45 ` Ilya Lipovsky 2007-09-07 17:54 ` Benedict, Michael ` (2 more replies) 0 siblings, 3 replies; 12+ messages in thread From: Ilya Lipovsky @ 2007-09-07 17:45 UTC (permalink / raw) To: 'Benedict, Michael', linuxppc-embedded Yes, I am uncertain. If it isn't a major headache, I think you can try the newest version (I think it's 2.6.1) and see if you get some resolution. Your code looks correct to me, so if the kernel developers did their job correctly, the only potentially weak link is glibc. I think pthread priority inheritance is an independent concept, but you shouldn't care since you explicitly set priorities. My kernel is 2.6.21, so I am of no help here, unfortunately. By the way, kudos to you for making crosstool compile glibc with NPTL support, I had issues getting there (I have 2.4 with NPTL, but it is natively installed)! How did you manage to get around the intermediate's compiler's lack of "__thread" support issue? Please do share the outcome!! -Ilya -----Original Message----- From: linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org [mailto:linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org] On Behalf Of Benedict, Michael Sent: Friday, September 07, 2007 1:25 PM To: linuxppc-embedded@ozlabs.org Subject: RE: futex priority based wakeup Thank you for the help! Same result with fflush :(. Are you maybe wrong because you have tested 2.5 and it doesn't work? Or just uncertain? The only thing I have read that mentions glibc version is the work Ingo did against 2.4 glibc and in the 2.6.16-mm1 patches... http://people.redhat.com/mingo/PI-futex-patches/. I can not detrmine how this relates to the 2.6.22 priority based futex wakeup commit. I am currently fighting crosstool-0.43 to build a newer glibc and I will share results if/when I get that to work. -Michael Ilya Lipovsky wrote: > ...Or maybe I am wrong :). Could you put fflush(stdout) after > each printf, > just to be completely certain that it misbehaves? > > -----Original Message----- > From: linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org > [mailto:linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.or g] On > Behalf Of Ilya Lipovsky Sent: Friday, September 07, 2007 12:41 PM > To: 'Benedict, Michael'; linuxppc-embedded@ozlabs.org > Subject: RE: futex priority based wakeup > > Looks like it is a libc issue. Apparently, GNU libc supports priority > futexes only in version 2.5 and higher. > > -----Original Message----- > From: linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org > [mailto:linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.or g] On > Behalf Of Benedict, Michael Sent: Friday, September 07, 2007 11:02 AM > To: linuxppc-embedded@ozlabs.org > Subject: futex priority based wakeup > > I recently upgraded to 2.6.22 to get the priority based futex wakeup > behavior. However, when I run a simple program (see below), based on > either a pthread_mutex_t or a sem_t, it seems that threads > are woken up > in FIFO order. I am using glibc 2.3.6 with NPTL and TLS, based off > crossdev-0.43. Could someone help me get priority-based > wakeup or point > me to a better place to get this question answered? Thank you, > Michael > > Code: > > pthread_mutex_t mymutex = PTHREAD_MUTEX_INITIALIZER; > > void *important(void *ign) > { > sleep(1); > printf("important waiting for mutex\n"); > if(pthread_mutex_lock(&mymutex)) { > perror("sem_wait"); > exit(1); > } else { > printf("important got mutex!\n"); > pthread_mutex_unlock(&mymutex); > } > > return NULL; > } > > > void *unimportant(void *ign) > { > printf("unimportant waiting for mutex\n"); > if(pthread_mutex_lock(&mymutex)) { > perror("sem_wait"); > exit(1); > } else { > printf("unimportant got mutex!\n"); > pthread_mutex_unlock(&mymutex); > } > > return NULL; > } > > int main() > { > struct sched_param p; > pthread_attr_t attr; > pthread_t i, u; > > pthread_mutex_lock(&mymutex); > > p.__sched_priority = sched_get_priority_min(SCHED_FIFO); > if(-1 == p.__sched_priority) { > perror("sched_get_priority_min"); > return 1; > } > pthread_attr_init(&attr); > pthread_attr_setschedpolicy(&attr, SCHED_FIFO); > pthread_attr_setschedparam(&attr, &p); > pthread_create(&u, &attr, unimportant, NULL); > > p.__sched_priority = sched_get_priority_max(SCHED_FIFO); > pthread_attr_setschedparam(&attr, &p); > pthread_create(&i, &attr, important, NULL); > > sleep(5); > printf("main unlocking mutex\n"); > pthread_mutex_unlock(&mymutex); > > pthread_join(u, NULL); > pthread_join(i, NULL); > > return 0; > } > > Which produces: > unimportant waiting for mutex > important waiting for mutex > main unlocking mutex > unimportant got mutex! > important got mutex! > > _______________________________________________ > Linuxppc-embedded mailing list > Linuxppc-embedded@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-embedded > > _______________________________________________ > Linuxppc-embedded mailing list > Linuxppc-embedded@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-embedded _______________________________________________ Linuxppc-embedded mailing list Linuxppc-embedded@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-embedded ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: futex priority based wakeup 2007-09-07 17:45 ` Ilya Lipovsky @ 2007-09-07 17:54 ` Benedict, Michael 2007-09-10 18:51 ` Benedict, Michael 2007-09-10 21:41 ` Benedict, Michael 2 siblings, 0 replies; 12+ messages in thread From: Benedict, Michael @ 2007-09-07 17:54 UTC (permalink / raw) To: linuxppc-embedded I am trying to build 2.6.1 as we speak. I don't think I said priority inheritance? If I did, sorry, must have been confused. As far as crosstool-0.43 and NTPL... I hate to be anti-climactic, but I didn't really have any issues. I build with gcc-3.4.5-glibc-2.3.6-nptl.dat, which contained: BINUTILS_DIR=3Dbinutils-2.15 GCC_DIR=3Dgcc-3.4.5 GLIBC_DIR=3Dglibc-2.3.6 LINUX_DIR=3Dlinux-2.6.15.4 LINUX_SANITIZED_HEADER_DIR=3Dlinux-libc-headers-2.6.12.0 GLIBCTHREADS_FILENAME=3Dglibc-linuxthreads-2.3.6 GLIBC_EXTRA_CONFIG=3D"$GLIBC_EXTRA_CONFIG --with-tls --with-__thread --enable-kernel=3D2.4.18" GLIBC_ADDON_OPTIONS=3D"=3Dnptl =3Dnptlonly" I will continue with updates as they happen. Thank you again, Michael Ilya Lipovsky wrote: > Yes, I am uncertain. If it isn't a major headache, I think > you can try the > newest version (I think it's 2.6.1) and see if you get some > resolution.=20 >=20 > Your code looks correct to me, so if the kernel developers > did their job > correctly, the only potentially weak link is glibc. >=20 > I think pthread priority inheritance is an independent > concept, but you > shouldn't care since you explicitly set priorities. >=20 > My kernel is 2.6.21, so I am of no help here, unfortunately. >=20 > By the way, kudos to you for making crosstool compile glibc with NPTL > support, I had issues getting there (I have 2.4 with NPTL, but it is > natively installed)! How did you manage to get around the > intermediate's compiler's lack of "__thread" support issue? >=20 > Please do share the outcome!! >=20 > -Ilya >=20 > -----Original Message----- > From: linuxppc-embedded-bounces+lipovsky=3Dcs.bu.edu@ozlabs.org > [mailto:linuxppc-embedded-bounces+lipovsky=3Dcs.bu.edu@ozlabs.or g] On > Behalf Of Benedict, Michael Sent: Friday, September 07, 2007 1:25 PM > To: linuxppc-embedded@ozlabs.org > Subject: RE: futex priority based wakeup >=20 > Thank you for the help! >=20 > Same result with fflush :(. >=20 > Are you maybe wrong because you have tested 2.5 and it > doesn't work? Or > just uncertain? >=20 > The only thing I have read that mentions glibc version is the > work Ingo > did against 2.4 glibc and in the 2.6.16-mm1 patches... > http://people.redhat.com/mingo/PI-futex-patches/. I can not detrmine > how this relates to the 2.6.22 priority based futex wakeup commit. >=20 > I am currently fighting crosstool-0.43 to build a newer glibc > and I will > share results if/when I get that to work. > -Michael >=20 > Ilya Lipovsky wrote: >> ...Or maybe I am wrong :). Could you put fflush(stdout) after >> each printf, >> just to be completely certain that it misbehaves? >>=20 >> -----Original Message----- >> From: linuxppc-embedded-bounces+lipovsky=3Dcs.bu.edu@ozlabs.org >> [mailto:linuxppc-embedded-bounces+lipovsky=3Dcs.bu.edu@ozlabs.or g] = On >> Behalf Of Ilya Lipovsky Sent: Friday, September 07, 2007 12:41 PM >> To: 'Benedict, Michael'; linuxppc-embedded@ozlabs.org >> Subject: RE: futex priority based wakeup >>=20 >> Looks like it is a libc issue. Apparently, GNU libc supports priority >> futexes only in version 2.5 and higher. >>=20 >> -----Original Message----- >> From: linuxppc-embedded-bounces+lipovsky=3Dcs.bu.edu@ozlabs.org >> [mailto:linuxppc-embedded-bounces+lipovsky=3Dcs.bu.edu@ozlabs.or g] = On >> Behalf Of Benedict, Michael Sent: Friday, September 07, 2007 11:02 AM >> To: linuxppc-embedded@ozlabs.org >> Subject: futex priority based wakeup >>=20 >> I recently upgraded to 2.6.22 to get the priority based futex wakeup >> behavior. However, when I run a simple program (see below), based on >> either a pthread_mutex_t or a sem_t, it seems that threads >> are woken up >> in FIFO order. I am using glibc 2.3.6 with NPTL and TLS, based off >> crossdev-0.43. Could someone help me get priority-based >> wakeup or point >> me to a better place to get this question answered? Thank you, >> Michael=20 >>=20 >> Code: >>=20 >> pthread_mutex_t mymutex =3D PTHREAD_MUTEX_INITIALIZER; >>=20 >> void *important(void *ign) >> { >> sleep(1); >> printf("important waiting for mutex\n"); >> if(pthread_mutex_lock(&mymutex)) { >> perror("sem_wait"); >> exit(1); >> } else { >> printf("important got mutex!\n"); >> pthread_mutex_unlock(&mymutex); >> } >>=20 >> return NULL; >> } >>=20 >>=20 >> void *unimportant(void *ign) >> { >> printf("unimportant waiting for mutex\n"); >> if(pthread_mutex_lock(&mymutex)) { >> perror("sem_wait"); >> exit(1); >> } else { >> printf("unimportant got mutex!\n"); >> pthread_mutex_unlock(&mymutex); >> } >>=20 >> return NULL; >> } >>=20 >> int main() >> { >> struct sched_param p; >> pthread_attr_t attr; >> pthread_t i, u; >>=20 >> pthread_mutex_lock(&mymutex); >>=20 >> p.__sched_priority =3D sched_get_priority_min(SCHED_FIFO); >> if(-1 =3D=3D p.__sched_priority) { >> perror("sched_get_priority_min"); >> return 1; >> } >> pthread_attr_init(&attr); >> pthread_attr_setschedpolicy(&attr, SCHED_FIFO); >> pthread_attr_setschedparam(&attr, &p); >> pthread_create(&u, &attr, unimportant, NULL); >>=20 >> p.__sched_priority =3D sched_get_priority_max(SCHED_FIFO); >> pthread_attr_setschedparam(&attr, &p); >> pthread_create(&i, &attr, important, NULL); >>=20 >> sleep(5); >> printf("main unlocking mutex\n"); >> pthread_mutex_unlock(&mymutex); >>=20 >> pthread_join(u, NULL); >> pthread_join(i, NULL); >>=20 >> return 0; >> } >>=20 >> Which produces: >> unimportant waiting for mutex >> important waiting for mutex >> main unlocking mutex >> unimportant got mutex! >> important got mutex! >>=20 >> _______________________________________________ >> Linuxppc-embedded mailing list >> Linuxppc-embedded@ozlabs.org >> https://ozlabs.org/mailman/listinfo/linuxppc-embedded >>=20 >> _______________________________________________ >> Linuxppc-embedded mailing list >> Linuxppc-embedded@ozlabs.org >> https://ozlabs.org/mailman/listinfo/linuxppc-embedded >=20 >=20 > _______________________________________________ > Linuxppc-embedded mailing list > Linuxppc-embedded@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-embedded ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: futex priority based wakeup 2007-09-07 17:45 ` Ilya Lipovsky 2007-09-07 17:54 ` Benedict, Michael @ 2007-09-10 18:51 ` Benedict, Michael 2007-09-10 21:41 ` Benedict, Michael 2 siblings, 0 replies; 12+ messages in thread From: Benedict, Michael @ 2007-09-10 18:51 UTC (permalink / raw) To: linuxppc-embedded Ilya Lipovsky wrote: > Yes, I am uncertain. If it isn't a major headache, I think > you can try the > newest version (I think it's 2.6.1) and see if you get some > resolution.=20 <snip> > Please do share the outcome!! >=20 > -Ilya No luck with 2.6.1: ~ $ /lib/libc.so.6 GNU C Library stable release version 2.6.1, by Roland McGrath et al. Copyright (C) 2007 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Compiled by GNU CC version 4.1.1. Compiled on a Linux 2.6.12 system on 2007-09-10. Available extensions: crypt add-on version 2.1 by Michael Glad and others Native POSIX Threads Library by Ulrich Drepper et al BIND-8.2.3-T5B For bug reporting instructions, please see: <http://www.gnu.org/software/libc/bugs.html>. ~ $ ./bar unimportant waiting for mutex important waiting for mutex main unlocking mutex unimportant got mutex! important got mutex! I will continue investigating and follow up on this thread if I come across anything. -Michael ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: futex priority based wakeup 2007-09-07 17:45 ` Ilya Lipovsky 2007-09-07 17:54 ` Benedict, Michael 2007-09-10 18:51 ` Benedict, Michael @ 2007-09-10 21:41 ` Benedict, Michael 2007-09-11 22:59 ` Ilya Lipovsky 2 siblings, 1 reply; 12+ messages in thread From: Benedict, Michael @ 2007-09-10 21:41 UTC (permalink / raw) To: linuxppc-embedded Ilya Lipovsky wrote: > Your code looks correct to me, so if the kernel developers > did their job > correctly, the only potentially weak link is glibc. >=20 Well, either the kernel developers didn't do their job, or I am missing something. The following also fails, and it should be bypassing glibc: #define _XOPEN_SOURCE 600 #include <linux/futex.h> #include <sys/time.h> #include <asm/atomic.h> #include <sys/syscall.h> #include <unistd.h> #include <sched.h> #include <stdint.h> #include <stdio.h> #include <stdlib.h> int myfutex =3D 0; void *important(void *ign) { sleep(1); printf("important waiting for futex\n"); fflush(stdout); if(syscall(SYS_futex, &myfutex, FUTEX_WAIT, 0, NULL)) { perror("futex"); exit(1); } else { printf("important got futex!\n"); fflush(stdout); syscall(SYS_futex, &myfutex, FUTEX_WAKE, 1, NULL); } return NULL; } void *unimportant(void *ign) { printf("unimportant waiting for futex\n"); fflush(stdout); if(syscall(SYS_futex, &myfutex, FUTEX_WAIT, 0, NULL)) { perror("futex"); exit(1); } else { printf("unimportant got futex!\n"); fflush(stdout); syscall(SYS_futex, &myfutex, FUTEX_WAKE, 1, NULL); } return NULL; } int main() { struct sched_param p; pthread_attr_t attr; pthread_t i, u; p.__sched_priority =3D sched_get_priority_min(SCHED_FIFO); if(-1 =3D=3D p.__sched_priority) { perror("sched_get_priority_min"); return 1; } pthread_attr_init(&attr); pthread_attr_setschedpolicy(&attr, SCHED_FIFO); pthread_attr_setschedparam(&attr, &p); pthread_create(&u, &attr, unimportant, NULL); p.__sched_priority =3D sched_get_priority_max(SCHED_FIFO); pthread_attr_setschedparam(&attr, &p); pthread_create(&i, &attr, important, NULL); sleep(5); printf("futex FUTEX_WAKE\n"); fflush(stdout); syscall(SYS_futex, &myfutex, FUTEX_WAKE, 1, NULL); pthread_join(u, NULL); pthread_join(i, NULL); return 0; } Which produces: unimportant waiting for futex important waiting for futex futex FUTEX_WAKE unimportant got futex! important got futex! Could someone with 2.6.22 please verify? =20 ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: futex priority based wakeup 2007-09-10 21:41 ` Benedict, Michael @ 2007-09-11 22:59 ` Ilya Lipovsky 2007-09-12 0:14 ` Nguyen Nguyen 0 siblings, 1 reply; 12+ messages in thread From: Ilya Lipovsky @ 2007-09-11 22:59 UTC (permalink / raw) To: 'Benedict, Michael', linuxppc-embedded Hmm. Just for kicks - inside the important thread could you add: int curpolicy; struct sched_param sp; pthread_getschedparam (pthread_self (), &curpolicy, &sp) printf("important's policy is %d and priority is %d\n", curpolicy, sp.__sched_priority); before the very first futex syscall and after your "printf("important got futex!\n");" line. Do similar for the unimportant thread, and see if you get anything weird - e.g. priorities come out to be the same for threads. -----Original Message----- From: linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org [mailto:linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org] On Behalf Of Benedict, Michael Sent: Monday, September 10, 2007 5:41 PM To: linuxppc-embedded@ozlabs.org Subject: RE: futex priority based wakeup Ilya Lipovsky wrote: > Your code looks correct to me, so if the kernel developers > did their job > correctly, the only potentially weak link is glibc. > Well, either the kernel developers didn't do their job, or I am missing something. The following also fails, and it should be bypassing glibc: #define _XOPEN_SOURCE 600 #include <linux/futex.h> #include <sys/time.h> #include <asm/atomic.h> #include <sys/syscall.h> #include <unistd.h> #include <sched.h> #include <stdint.h> #include <stdio.h> #include <stdlib.h> int myfutex = 0; void *important(void *ign) { sleep(1); printf("important waiting for futex\n"); fflush(stdout); if(syscall(SYS_futex, &myfutex, FUTEX_WAIT, 0, NULL)) { perror("futex"); exit(1); } else { printf("important got futex!\n"); fflush(stdout); syscall(SYS_futex, &myfutex, FUTEX_WAKE, 1, NULL); } return NULL; } void *unimportant(void *ign) { printf("unimportant waiting for futex\n"); fflush(stdout); if(syscall(SYS_futex, &myfutex, FUTEX_WAIT, 0, NULL)) { perror("futex"); exit(1); } else { printf("unimportant got futex!\n"); fflush(stdout); syscall(SYS_futex, &myfutex, FUTEX_WAKE, 1, NULL); } return NULL; } int main() { struct sched_param p; pthread_attr_t attr; pthread_t i, u; p.__sched_priority = sched_get_priority_min(SCHED_FIFO); if(-1 == p.__sched_priority) { perror("sched_get_priority_min"); return 1; } pthread_attr_init(&attr); pthread_attr_setschedpolicy(&attr, SCHED_FIFO); pthread_attr_setschedparam(&attr, &p); pthread_create(&u, &attr, unimportant, NULL); p.__sched_priority = sched_get_priority_max(SCHED_FIFO); pthread_attr_setschedparam(&attr, &p); pthread_create(&i, &attr, important, NULL); sleep(5); printf("futex FUTEX_WAKE\n"); fflush(stdout); syscall(SYS_futex, &myfutex, FUTEX_WAKE, 1, NULL); pthread_join(u, NULL); pthread_join(i, NULL); return 0; } Which produces: unimportant waiting for futex important waiting for futex futex FUTEX_WAKE unimportant got futex! important got futex! Could someone with 2.6.22 please verify? _______________________________________________ Linuxppc-embedded mailing list Linuxppc-embedded@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-embedded ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: futex priority based wakeup 2007-09-11 22:59 ` Ilya Lipovsky @ 2007-09-12 0:14 ` Nguyen Nguyen 2007-09-12 1:09 ` Ilya Lipovsky 2007-09-12 14:56 ` Benedict, Michael 0 siblings, 2 replies; 12+ messages in thread From: Nguyen Nguyen @ 2007-09-12 0:14 UTC (permalink / raw) To: Ilya Lipovsky; +Cc: Benedict, Michael, linuxppc-embedded [-- Attachment #1: Type: text/plain, Size: 4038 bytes --] I have seen something similar before. Our fix was to use pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED) so child threads wouldn't inherit attribute from parent. Hope it helps. On 9/11/07, Ilya Lipovsky <lipovsky@cs.bu.edu> wrote: > > Hmm. Just for kicks - inside the important thread could you add: > > int curpolicy; > struct sched_param sp; > pthread_getschedparam (pthread_self (), &curpolicy, &sp) > printf("important's policy is %d and priority is %d\n", curpolicy, > sp.__sched_priority); > > before the very first futex syscall and after your "printf("important got > futex!\n");" line. > > Do similar for the unimportant thread, and see if you get anything weird - > e.g. priorities come out to be the same for threads. > > > -----Original Message----- > From: linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org > [mailto:linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org] On Behalf > Of Benedict, Michael > Sent: Monday, September 10, 2007 5:41 PM > To: linuxppc-embedded@ozlabs.org > Subject: RE: futex priority based wakeup > > Ilya Lipovsky wrote: > > Your code looks correct to me, so if the kernel developers > > did their job > > correctly, the only potentially weak link is glibc. > > > > Well, either the kernel developers didn't do their job, or I am missing > something. The following also fails, and it should be bypassing glibc: > > #define _XOPEN_SOURCE 600 > > #include <linux/futex.h> > #include <sys/time.h> > #include <asm/atomic.h> > > #include <sys/syscall.h> > #include <unistd.h> > > #include <sched.h> > #include <stdint.h> > #include <stdio.h> > #include <stdlib.h> > > int myfutex = 0; > > void *important(void *ign) > { > sleep(1); > printf("important waiting for futex\n"); > fflush(stdout); > if(syscall(SYS_futex, &myfutex, FUTEX_WAIT, 0, NULL)) { > perror("futex"); > exit(1); > } else { > printf("important got futex!\n"); > fflush(stdout); > syscall(SYS_futex, &myfutex, FUTEX_WAKE, 1, NULL); > } > > return NULL; > } > > > void *unimportant(void *ign) > { > printf("unimportant waiting for futex\n"); > fflush(stdout); > if(syscall(SYS_futex, &myfutex, FUTEX_WAIT, 0, NULL)) { > perror("futex"); > exit(1); > } else { > printf("unimportant got futex!\n"); > fflush(stdout); > syscall(SYS_futex, &myfutex, FUTEX_WAKE, 1, NULL); > } > > return NULL; > } > > int main() > { > struct sched_param p; > pthread_attr_t attr; > pthread_t i, u; > > p.__sched_priority = sched_get_priority_min(SCHED_FIFO); > if(-1 == p.__sched_priority) { > perror("sched_get_priority_min"); > return 1; > } > pthread_attr_init(&attr); > pthread_attr_setschedpolicy(&attr, SCHED_FIFO); > pthread_attr_setschedparam(&attr, &p); > pthread_create(&u, &attr, unimportant, NULL); > > p.__sched_priority = sched_get_priority_max(SCHED_FIFO); > pthread_attr_setschedparam(&attr, &p); > pthread_create(&i, &attr, important, NULL); > > sleep(5); > printf("futex FUTEX_WAKE\n"); > fflush(stdout); > syscall(SYS_futex, &myfutex, FUTEX_WAKE, 1, NULL); > > pthread_join(u, NULL); > pthread_join(i, NULL); > > return 0; > } > > Which produces: > unimportant waiting for futex > important waiting for futex > futex FUTEX_WAKE > unimportant got futex! > important got futex! > > > Could someone with 2.6.22 please verify? > > _______________________________________________ > Linuxppc-embedded mailing list > Linuxppc-embedded@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-embedded > > _______________________________________________ > Linuxppc-embedded mailing list > Linuxppc-embedded@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-embedded > > [-- Attachment #2: Type: text/html, Size: 7379 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: futex priority based wakeup 2007-09-12 0:14 ` Nguyen Nguyen @ 2007-09-12 1:09 ` Ilya Lipovsky 2007-09-12 14:56 ` Benedict, Michael 1 sibling, 0 replies; 12+ messages in thread From: Ilya Lipovsky @ 2007-09-12 1:09 UTC (permalink / raw) To: 'Nguyen Nguyen'; +Cc: 'Benedict, Michael', linuxppc-embedded [-- Attachment #1: Type: text/plain, Size: 4306 bytes --] Good info. Indeed, looks like this is what glibc actually does: http://www.cygwin.com/ml/libc-alpha/2007-06/msg00097.html _____ From: tinghich@gmail.com [mailto:tinghich@gmail.com] On Behalf Of Nguyen Nguyen Sent: Tuesday, September 11, 2007 8:15 PM To: Ilya Lipovsky Cc: Benedict, Michael; linuxppc-embedded@ozlabs.org Subject: Re: futex priority based wakeup I have seen something similar before. Our fix was to use pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED) so child threads wouldn't inherit attribute from parent. Hope it helps. On 9/11/07, Ilya Lipovsky <lipovsky@cs.bu.edu> wrote: Hmm. Just for kicks - inside the important thread could you add: int curpolicy; struct sched_param sp; pthread_getschedparam (pthread_self (), &curpolicy, &sp) printf("important's policy is %d and priority is %d\n", curpolicy, sp.__sched_priority); before the very first futex syscall and after your "printf("important got futex!\n");" line. Do similar for the unimportant thread, and see if you get anything weird - e.g. priorities come out to be the same for threads. -----Original Message----- From: linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org [mailto: <mailto:linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org> linuxppc-embedded-bounces+lipovsky=cs.bu.edu@ozlabs.org] On Behalf Of Benedict, Michael Sent: Monday, September 10, 2007 5:41 PM To: linuxppc-embedded@ozlabs.org Subject: RE: futex priority based wakeup Ilya Lipovsky wrote: > Your code looks correct to me, so if the kernel developers > did their job > correctly, the only potentially weak link is glibc. > Well, either the kernel developers didn't do their job, or I am missing something. The following also fails, and it should be bypassing glibc: #define _XOPEN_SOURCE 600 #include <linux/futex.h> #include <sys/time.h> #include <asm/atomic.h> #include <sys/syscall.h> #include <unistd.h> #include <sched.h> #include <stdint.h> #include <stdio.h> #include <stdlib.h> int myfutex = 0; void *important(void *ign) { sleep(1); printf("important waiting for futex\n"); fflush(stdout); if(syscall(SYS_futex, &myfutex, FUTEX_WAIT, 0, NULL)) { perror("futex"); exit(1); } else { printf("important got futex!\n"); fflush(stdout); syscall(SYS_futex, &myfutex, FUTEX_WAKE, 1, NULL); } return NULL; } void *unimportant(void *ign) { printf("unimportant waiting for futex\n"); fflush(stdout); if(syscall(SYS_futex, &myfutex, FUTEX_WAIT, 0, NULL)) { perror("futex"); exit(1); } else { printf("unimportant got futex!\n"); fflush(stdout); syscall(SYS_futex, &myfutex, FUTEX_WAKE, 1, NULL); } return NULL; } int main() { struct sched_param p; pthread_attr_t attr; pthread_t i, u; p.__sched_priority = sched_get_priority_min(SCHED_FIFO); if(-1 == p.__sched_priority) { perror("sched_get_priority_min"); return 1; } pthread_attr_init(&attr); pthread_attr_setschedpolicy(&attr, SCHED_FIFO); pthread_attr_setschedparam(&attr, &p); pthread_create(&u, &attr, unimportant, NULL); p.__sched_priority = sched_get_priority_max(SCHED_FIFO); pthread_attr_setschedparam(&attr, &p); pthread_create(&i, &attr, important, NULL); sleep(5); printf("futex FUTEX_WAKE\n"); fflush(stdout); syscall(SYS_futex, &myfutex, FUTEX_WAKE, 1, NULL); pthread_join(u, NULL); pthread_join(i, NULL); return 0; } Which produces: unimportant waiting for futex important waiting for futex futex FUTEX_WAKE unimportant got futex! important got futex! Could someone with 2.6.22 please verify? _______________________________________________ Linuxppc-embedded mailing list Linuxppc-embedded@ozlabs.org <mailto:Linuxppc-embedded@ozlabs.org> https://ozlabs.org/mailman/listinfo/linuxppc-embedded _______________________________________________ Linuxppc-embedded mailing list Linuxppc-embedded@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-embedded [-- Attachment #2: Type: text/html, Size: 11905 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: futex priority based wakeup 2007-09-12 0:14 ` Nguyen Nguyen 2007-09-12 1:09 ` Ilya Lipovsky @ 2007-09-12 14:56 ` Benedict, Michael 1 sibling, 0 replies; 12+ messages in thread From: Benedict, Michael @ 2007-09-12 14:56 UTC (permalink / raw) To: linuxppc-embedded Nguyen Nguyen wrote: > I have seen something similar before. Our fix was to use > pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED) > so child threads wouldn't inherit attribute from parent. > On 9/11/07, Ilya Lipovsky <lipovsky@cs.bu.edu> wrote: >=20 > Hmm. Just for kicks - inside the important thread could you add: <snip> > printf("important's policy is %d and priority is %d\n", curpolicy, <snip> > Do similar for the unimportant thread, and see if you > get anything weird - > e.g. priorities come out to be the same for threads. Thank you both! You were each correct, and your input has been greatly appreciated. Adding the printf() resulted in: unimportant's policy is 0 and priority is 0 unimportant waiting for futex important's policy is 0 and priority is 0 important waiting for futex main's policy is 0 and priority is 0 futex FUTEX_WAKE unimportant's policy is 0 and priority is 0 unimportant got futex! important's policy is 0 and priority is 0 important got futex! Adding the PTHREAD_EXPLICIT_SCHED resulted in: unimportant's policy is 1 and priority is 1 unimportant waiting for futex important's policy is 1 and priority is 99 important waiting for futex main's policy is 0 and priority is 0 futex FUTEX_WAKE important's policy is 1 and priority is 99 important got futex! unimportant's policy is 1 and priority is 1 unimportant got futex! Verifying with pthread_muxtex / glibc 2.3.6: unimportant's policy is 1 and priority is 1 unimportant waiting for mutex important's policy is 1 and priority is 99 important waiting for mutex main's policy is 0 and priority is 0 main unlocking mutex important's policy is 1 and priority is 99 important got mutex! unimportant's policy is 1 and priority is 1 unimportant got mutex! And, to come full circle, regression on 2.6.21 resulted in: unimportant's policy is 1 and priority is 1 unimportant waiting for futex important's policy is 1 and priority is 99 important waiting for futex main's policy is 0 and priority is 0 futex FUTEX_WAKE unimportant's policy is 1 and priority is 1 unimportant got futex! important's policy is 1 and priority is 99 important got futex! Now it is time to audit my other code and make sure to add PTHREAD_EXPLICIT_SCHED where appropriate. Thank you. -Michael ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2007-09-12 14:57 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2007-09-07 15:02 futex priority based wakeup Benedict, Michael 2007-09-07 16:41 ` Ilya Lipovsky 2007-09-07 17:16 ` Ilya Lipovsky 2007-09-07 17:24 ` Benedict, Michael 2007-09-07 17:45 ` Ilya Lipovsky 2007-09-07 17:54 ` Benedict, Michael 2007-09-10 18:51 ` Benedict, Michael 2007-09-10 21:41 ` Benedict, Michael 2007-09-11 22:59 ` Ilya Lipovsky 2007-09-12 0:14 ` Nguyen Nguyen 2007-09-12 1:09 ` Ilya Lipovsky 2007-09-12 14:56 ` Benedict, Michael
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).