Cobol app takes 100% CPU

Cobol app takes 100% CPU

[Rodrigo Real]

Hello,

I am quiet here for sometime, so I am not exactly a new user of the
list, but a quiet one...

I've been using dosemu+freedos for a Cobol application for 4 or 5
years, we have a dedicated server for that aplication and everybody
rsh to it to run the application. In the begining we had 10 to 12
users, now we have 26.

This cobol app always tried to eat the most of cpu it can, with 10 or
12 users the situation were acceptable, but now with 26 it is been
very hard to work.

I tried many approaches, the computer is a AMD 64 running a 2.6.16
kernel (tried many kernels also), dosemu 1.2.2-8. I swapped an IDE
disk for two SATA in stripping. But nothing...

It seems that the cobol app keeps reading the keyboard all the time, I
talked to the app programmer and he said they use the ACCEPT call,
they don't do the loop, maybe accept does... I don't know. The funny
thing is that in Windows it just doesn't happen, the cpu stays in 1%
of usage.

My guess is that the problem is with some emulation that dosemu  does,
but I don't know what to think anymore.

Does anybody have any ideia?

Thanks,
Rodrigo

Re: Cobol app takes 100% CPU

[Frank Cox]

On Wed, 26 Mar 2008 09:55:20 -0300
Rodrigo Real <rreal@ucpel.tche.br> wrote:

> This cobol app always tried to eat the most of cpu it can, with 10 or
> 12 users the situation were acceptable, but now with 26 it is been
> very hard to work.


Have you tried Tame?

-- 
MELVILLE THEATRE ~ Melville Sask ~ http://www.melvilletheatre.com

Re: Cobol app takes 100% CPU

[Rodrigo Real]

Hi Frank

Frank Cox <theatre@sasktel.net> writes:

> On Wed, 26 Mar 2008 09:55:20 -0300
> Rodrigo Real <rreal@ucpel.tche.br> wrote:
>
>> This cobol app always tried to eat the most of cpu it can, with 10 or
>> 12 users the situation were acceptable, but now with 26 it is been
>> very hard to work.
>
>
> Have you tried Tame?

I once took a look at Tame, but didn't tried it, since it was a paid
software and I didn't know anything about it, I just didn't pay
attention. But it is good to know that it works.

When you answered my question I was in the middle of a solution and
followed it until the end.

I decided to hack dosemu code to limit its cpu usage. I made some
tests that were very fine, and found a solution for me.

In the end I changed only two lines of code. In src/include/timers.h I
multiplied by 1000 the JIFFIE_TIME, so it became:

-#define JIFFIE_TIME            (1000000/sysconf(_SC_CLK_TCK))
+#define JIFFIE_TIME            (1000000000/sysconf(_SC_CLK_TCK))

And in src/base/bios/int16.c I added a usleep before
check_key_available call:

   case 0x11:
+    usleep(INT2F_IDLE_USECS/100000);
     check_key_available(EXTENDED);
     break;

Now the load of the server is less them one, even with 20 dosemu
processes running, before that changes the load would be in 12. And
the users are very happy with the performance.

Don't know if this is a good forum for that, but I guess that this
sort of thing should be a parameter, so users can calibrate their
speed. Maybe I am missing something, but I guess that many dosemu
users would be in a situation similar to mine.

Thanks for helping!

Cheers,
Rodrigo

Re: Cobol app takes 100% CPU

[Frank Cox]

On Thu, 27 Mar 2008 09:11:33 -0300
Rodrigo Real <rreal@ucpel.tche.br> wrote:

> Don't know if this is a good forum for that, but I guess that this
> sort of thing should be a parameter, so users can calibrate their
> speed. Maybe I am missing something, but I guess that many dosemu
> users would be in a situation similar to mine.

As long as it has no weird side-effects, it should definitely be added to
dosemu, in my humble opinion.  Even if it does, in fact -- as long as it's an
adjustable option that can be switched on and off.

-- 
MELVILLE THEATRE ~ Melville Sask ~ http://www.melvilletheatre.com

Re: Cobol app takes 100% CPU

[Bart Oldeman]

On 27/03/2008, Rodrigo Real <rreal@ucpel.tche.br> wrote:
>  And in src/base/bios/int16.c I added a usleep before
>  check_key_available call:
>
>    case 0x11:
>  +    usleep(INT2F_IDLE_USECS/100000);
>      check_key_available(EXTENDED);
>      break;

Good that that works for you, but I'd like to ask:
1. You said that the problem occurs with 1.2.2-8. Does it also occur
with DOSEMU 1.4.0?
The jiffies adjustment does not affect 1.4.0 anymore.
2. check_key_available has some code itself to make DOSEMU to become
idle. Did you play with the numbers in those? They are in the
idle(500, 20, 0, INT2F_IDLE_USECS, "int16");
call: lower numbers mean quicker sleeping.

Bart

>
>  Now the load of the server is less them one, even with 20 dosemu
>  processes running, before that changes the load would be in 12. And
>  the users are very happy with the performance.
>
>  Don't know if this is a good forum for that, but I guess that this
>  sort of thing should be a parameter, so users can calibrate their
>  speed. Maybe I am missing something, but I guess that many dosemu
>  users would be in a situation similar to mine.
>
>  Thanks for helping!
>
>  Cheers,
>
> Rodrigo
>
> --
>  To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
>  the body of a message to majordomo@vger.kernel.org
>  More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Cobol app takes 100% CPU

[Mike McCarty]

Rodrigo Real wrote:
> I once took a look at Tame, but didn't tried it, since it was a paid
> software and I didn't know anything about it, I just didn't pay
> attention. But it is good to know that it works.

I didn't reply, because I wasn't sure I understood your
issue. Have you tried simply running dosemu with nice?

I wouldn't care if dosemu ate all the CPU, so long as
other processes weren't slowed down when they wanted to
run.

For example, I run the GIMPS on my machine. It's always
100% CPU utilization as a consequence. But if the response
is slow, it's not because of GIMPS.

Mike
-- 
p="p=%c%s%c;main(){printf(p,34,p,34);}";main(){printf(p,34,p,34);}
Oppose globalization and One World Governments like the UN.
This message made from 100% recycled bits.
You have found the bank of Larn.
I can explain it for you, but I can't understand it for you.
I speak only for myself, and I am unanimous in that!

Re: Cobol app takes 100% CPU

[Rodrigo Real]

Hi Bart

"Bart Oldeman" <bartoldeman@users.sourceforge.net> writes:

> On 27/03/2008, Rodrigo Real <rreal@ucpel.tche.br> wrote:
>>  And in src/base/bios/int16.c I added a usleep before
>>  check_key_available call:
>>
>>    case 0x11:
>>  +    usleep(INT2F_IDLE_USECS/100000);
>>      check_key_available(EXTENDED);
>>      break;
>
> Good that that works for you, but I'd like to ask:
> 1. You said that the problem occurs with 1.2.2-8. Does it also occur
> with DOSEMU 1.4.0?

Yes with 1.4.0 the problem still ocurred, I did the changes in the
1.4.0 version, but before them the cpu usage were 100%.

> The jiffies adjustment does not affect 1.4.0 anymore.

I don't know much about dosemu implementation, so I don't know exactly
what jiffies does, but my it changes INT2F_IDLE_USECS,
INT15_IDLE_USECS and INT28_IDLE_USECS, since I used INT2F_IDLE_USECS
in the usleep, I guess that it changes its behaviour.

Now I see that I could had only divided INT2F_IDLE_USECS by 100 and
would have the same effect...

> 2. check_key_available has some code itself to make DOSEMU to become
> idle. Did you play with the numbers in those? They are in the
> idle(500, 20, 0, INT2F_IDLE_USECS, "int16");
> call: lower numbers mean quicker sleeping.

I didn't try that call (didn't knew about that).

I will make some more tests here with those information you sent, and
will see what results I can get. After that, I will send the results
to the list.

Rodrigo


>
> Bart
>
>>
>>  Now the load of the server is less them one, even with 20 dosemu
>>  processes running, before that changes the load would be in 12. And
>>  the users are very happy with the performance.
>>
>>  Don't know if this is a good forum for that, but I guess that this
>>  sort of thing should be a parameter, so users can calibrate their
>>  speed. Maybe I am missing something, but I guess that many dosemu
>>  users would be in a situation similar to mine.
>>
>>  Thanks for helping!
>>
>>  Cheers,
>>
>> Rodrigo
>>
>> --
>>  To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
>>  the body of a message to majordomo@vger.kernel.org
>>  More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Cobol app takes 100% CPU

[Rodrigo Real]

Mike McCarty <Mike.McCarty@sbcglobal.net> writes:

> Rodrigo Real wrote:
>> I once took a look at Tame, but didn't tried it, since it was a paid
>> software and I didn't know anything about it, I just didn't pay
>> attention. But it is good to know that it works.
>
> I didn't reply, because I wasn't sure I understood your
> issue. Have you tried simply running dosemu with nice?

Yes, for a long time it solved the problems. But with the growth of
users of the application, I couldn't find any solution with nice,
mostly because everybody is "fighting" for the CPU in the same nice.

I also tried using different nices considering the usage profile of
each users. But it didn't work also, because the users doesn't behave
the same way all the time.

>
> I wouldn't care if dosemu ate all the CPU, so long as
> other processes weren't slowed down when they wanted to
> run.
>
> For example, I run the GIMPS on my machine. It's always
> 100% CPU utilization as a consequence. But if the response
> is slow, it's not because of GIMPS.

This works because GIMPS is with a high nice, and everything else is
not. You have two different application profiles, so nice fits
well. In my case dosemu is the most important application on this
machine.

Rodrigo

>
> Mike
> -- 
> p="p=%c%s%c;main(){printf(p,34,p,34);}";main(){printf(p,34,p,34);}
> Oppose globalization and One World Governments like the UN.
> This message made from 100% recycled bits.
> You have found the bank of Larn.
> I can explain it for you, but I can't understand it for you.
> I speak only for myself, and I am unanimous in that!
> --
> To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Cobol app takes 100% CPU

[Manfred Scherer]

Hi all,
I have had the the same problem with other programs i.e. dbase 3+ word 4.0 ...
in dosemu 1.3.4 I have made a small change in
src/base/bios/int16.c  in function check_key_available():

here is the code for dosemu 1.3.4, I think this is equal to 1.4.0:

static unsigned check_key_available(int extended)
{
  unsigned keyptr = get_key(extended);
  if(keyptr == -1) {
    if(!port60_buffer || (port60_buffer & 0x80))
      trigger_idle();
    else
      reset_idle(0);
    /* 2006-12-13
     * dbase3+, word4.0, .... CPU-load is to heavy during idle. --ms
     * idle(500, 20, 0, INT2F_IDLE_USECS, "int16");
     */
    idle(10, 20, 0, INT2F_IDLE_USECS, "int16");
  } else {
    reset_idle(1);
  }
  return get_key(extended);
}

I think that may help for Cobol too.

Manfred

Re: Cobol app takes 100% CPU

[Jan Kandziora]

[-- Attachment #1: Type: text/plain, Size: 604 bytes --]

Am Donnerstag, 27. März 2008 19:11 schrieb Rodrigo Real:
>
> Yes, for a long time it solved the problems. But with the growth of
> users of the application, I couldn't find any solution with nice,
> mostly because everybody is "fighting" for the CPU in the same nice.
>
In that case, upgrading to Linux-2.6.24 may help. It has scheduling groups 
where each user is granted equivalent part of the CPU anytime (root has 
doubled share). Haven't tested yet how it affects dosemu, though...

Kind regards

	Jan
-- 
BASIC is to computer programming as QWERTY is to typing.
		-- Seymour Papert

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Cobol app takes 100% CPU

[Rodrigo Real]

Hi Jan

Jan Kandziora <jjj@gmx.de> writes:

> Am Donnerstag, 27. März 2008 19:11 schrieb Rodrigo Real:
>>
>> Yes, for a long time it solved the problems. But with the growth of
>> users of the application, I couldn't find any solution with nice,
>> mostly because everybody is "fighting" for the CPU in the same nice.
>>
> In that case, upgrading to Linux-2.6.24 may help. It has scheduling groups 
> where each user is granted equivalent part of the CPU anytime (root has 
> doubled share). Haven't tested yet how it affects dosemu, though...

I tried that approach also, but it didn't got any better, I was
sucessful in dividing users in different queues, but some of them got
so low cpu that they couldn't use the application.

Regards,
Rodrigo

>
> Kind regards
>
> 	Jan
> -- 
> BASIC is to computer programming as QWERTY is to typing.
> 		-- Seymour Papert
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Cobol app takes 100% CPU

[Manfred Scherer]

Hi Rodrigo,

another candidate for some sort of problems is function msdos()
in ./src/base/async/int.c

I've made a small change for 'get time & date':

here is the code:

  case 0x2C: {                   /* get time & date */
      /*
       * 2004/08/20, 2006/11/25
       * waiting loops, like 'wait until time ...' call never a trigger_idle()
       * but 90% cpu load. --ms
       * idle(2, 100, 0, INT2F_IDLE_USECS, "dos_time");
       */
      idle(0, 20, 0, INT2F_IDLE_USECS, "dos_time");
      return 0;
    }

You can try this too, but I think this should not be a problem in 
your cobol application.  This is also a result of debugging my own apps.

Manfred
________________________________________________________________

Am Montag, 31. März 2008 15:46 schrieben Sie:
> 
> Hi Manfred
> 
> Changing idle first parameter to 10 didn't help, but changing it to 1
> or 2 slowed down the cobol app. I was changing e-mails with Bart
> privatetly about that. The problem with this approach is that when the
> app really needs the cpu, for generating reports, it also runs slow, I
> guess because it keeps updating a number on the screen.
> 
> After some debugging I identified an AX pattern when the app is in the
> menus, which is the main problem of this app, so I only slow down
> dosemu when I detect this pattern:
> 
> --- ../dosemu-1.4.0/src/base/bios/int16.c       2007-05-04
>     02:59:48.000000000 -0300
> +++ src/base/bios/int16.c       2008-03-28 15:10:13.000000000 -0300
> @@ -112,16 +112,22 @@
>  
>  static unsigned check_key_available(int extended)
>  {
> +  static int oldAX=0;
>    unsigned keyptr = get_key(extended);
>    if(keyptr == -1) {
>      if(!port60_buffer || (port60_buffer & 0x80))
>        trigger_idle();
>      else
>        reset_idle(0);
> -    idle(500, 20, 0, INT2F_IDLE_USECS, "int16");
> +/*     printf ("AX=%x\n",_AX); */
> +    if (_AX == 0x1100 && oldAX==0x1103 ) /* here I detect the pattern */
> +      idle(2, 20, 0, INT2F_IDLE_USECS, "int16");
> +    else
> +      idle(500, 20, 0, INT2F_IDLE_USECS, "int16");
>    } else {
>      reset_idle(1);
>    }
> +  oldAX = _AX;
>    return get_key(extended);
>  }
> 
> This is a sort of dirty solution, but it worked fine...
> 
> Thanks,
> Rodrigo
> 
> 
> 
> Manfred Scherer <manfred.scherer.mhm@t-online.de> writes:
> 
> > Hi all,
> > I have had the the same problem with other programs i.e. dbase 3+ word 4.0 ...
> > in dosemu 1.3.4 I have made a small change in
> > src/base/bios/int16.c  in function check_key_available():
> >
> > here is the code for dosemu 1.3.4, I think this is equal to 1.4.0:
> >
> > static unsigned check_key_available(int extended)
> > {
> >   unsigned keyptr = get_key(extended);
> >   if(keyptr == -1) {
> >     if(!port60_buffer || (port60_buffer & 0x80))
> >       trigger_idle();
> >     else
> >       reset_idle(0);
> >     /* 2006-12-13
> >      * dbase3+, word4.0, .... CPU-load is to heavy during idle. --ms
> >      * idle(500, 20, 0, INT2F_IDLE_USECS, "int16");
> >      */
> >     idle(10, 20, 0, INT2F_IDLE_USECS, "int16");
> >   } else {
> >     reset_idle(1);
> >   }
> >   return get_key(extended);
> > }
> >
> > I think that may help for Cobol too.
> >
> > Manfred
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Cobol app takes 100% CPU

[Rodrigo Real]

Hi Manfred

Changing idle first parameter to 10 didn't help, but changing it to 1
or 2 slowed down the cobol app. I was changing e-mails with Bart
privatetly about that. The problem with this approach is that when the
app really needs the cpu, for generating reports, it also runs slow, I
guess because it keeps updating a number on the screen.

After some debugging I identified an AX pattern when the app is in the
menus, which is the main problem of this app, so I only slow down
dosemu when I detect this pattern:

--- ../dosemu-1.4.0/src/base/bios/int16.c       2007-05-04
    02:59:48.000000000 -0300
+++ src/base/bios/int16.c       2008-03-28 15:10:13.000000000 -0300
@@ -112,16 +112,22 @@
 
 static unsigned check_key_available(int extended)
 {
+  static int oldAX=0;
   unsigned keyptr = get_key(extended);
   if(keyptr == -1) {
     if(!port60_buffer || (port60_buffer & 0x80))
       trigger_idle();
     else
       reset_idle(0);
-    idle(500, 20, 0, INT2F_IDLE_USECS, "int16");
+/*     printf ("AX=%x\n",_AX); */
+    if (_AX == 0x1100 && oldAX==0x1103 ) /* here I detect the pattern */
+      idle(2, 20, 0, INT2F_IDLE_USECS, "int16");
+    else
+      idle(500, 20, 0, INT2F_IDLE_USECS, "int16");
   } else {
     reset_idle(1);
   }
+  oldAX = _AX;
   return get_key(extended);
 }

This is a sort of dirty solution, but it worked fine...

Thanks,
Rodrigo



Manfred Scherer <manfred.scherer.mhm@t-online.de> writes:

> Hi all,
> I have had the the same problem with other programs i.e. dbase 3+ word 4.0 ...
> in dosemu 1.3.4 I have made a small change in
> src/base/bios/int16.c  in function check_key_available():
>
> here is the code for dosemu 1.3.4, I think this is equal to 1.4.0:
>
> static unsigned check_key_available(int extended)
> {
>   unsigned keyptr = get_key(extended);
>   if(keyptr == -1) {
>     if(!port60_buffer || (port60_buffer & 0x80))
>       trigger_idle();
>     else
>       reset_idle(0);
>     /* 2006-12-13
>      * dbase3+, word4.0, .... CPU-load is to heavy during idle. --ms
>      * idle(500, 20, 0, INT2F_IDLE_USECS, "int16");
>      */
>     idle(10, 20, 0, INT2F_IDLE_USECS, "int16");
>   } else {
>     reset_idle(1);
>   }
>   return get_key(extended);
> }
>
> I think that may help for Cobol too.
>
> Manfred
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html