Re: [KVM-AUTOTEST,01/17] Add new module kvm_subprocess

["Cao, Chen" <kcao@redhat.com>]

Hi, Michael,

I found that if the sessions initialized using kvm_subprcoess are not closed,
the processes will never exit, and /tmp/kvm_spawn will be filled with the
temporary files.

And we can find in the code,
# kvm_subprocess.py
...
        # Read from child and write to files/pipes
        while True:
            check_termination = False
            # Make a list of reader pipes whose buffers are not empty
            fds = [fd for (i, fd) in enumerate(reader_fds) if buffers[i]]
            # Wait until there's something to do
            r, w, x = select.select([shell_fd, inpipe_fd], fds, [], 0.5)
            # If a reader pipe is ready for writing --
            for (i, fd) in enumerate(reader_fds):
                if fd in w:
                    bytes_written = os.write(fd, buffers[i])
                    buffers[i] = buffers[i][bytes_written:]
            # If there's data to read from the child process --
            if shell_fd in r:
                try:
                    data = os.read(shell_fd, 16384)
                except OSError:
                    data = ""
                if not data:
                    check_termination = True
                # Remove carriage returns from the data -- they often cause
                # trouble and are normally not needed
                data = data.replace("\r", "")
                output_file.write(data)
                output_file.flush()
                for i in range(len(readers)):
                    buffers[i] += data
            # If os.read() raised an exception or there was nothing to read --
            if check_termination or shell_fd not in r:
                pid, status = os.waitpid(shell_pid, os.WNOHANG)
                if pid:
                    status = os.WEXITSTATUS(status)
                    break
            # If there's data to read from the client --
            if inpipe_fd in r:
                data = os.read(inpipe_fd, 1024)
                os.write(shell_fd, data)
...

that if session.close() is not called, we will loop in the 'while' forever.

So, user have to make sure that unnecessary sessions are all killed,
otherwise, running some testcase(s) for huge number of times will suck
out all the system resource, which I think is very inconvenient.
Especially when we have to take care of many exceptions that may be raised
by our program.

e.g.

...
session = kvm_test_utils.wait_for_login(vm)
...
session2 = kvm_test_utils.wait_for_login(vm_x)
...
try:
    ...
except ...:
    ...
...
(other code may raise exceptions)
...
try:
    ...
except ...:
    ...
...
try:
    ...
except ...:
    ...
...

cleaning up the sessions will be exhausting here.

Do we have a good (or better) way to handle this?
Thanks.


Regards,

Cao, Chen
2009-10-12

On Mon, Jul 20, 2009 at 12:07 PM, Michael Goldish<mgoldish@redhat.com>
wrote:
> This module is intended to be used for controlling all child
> processes in KVM
> tests: both QEMU processes and SSH/SCP/Telnet processes. Processes
> started with
> this module keep running and can be interacted with even after the
> parent
> process exits.
>
> The current run_bg() utility tracks a child process as long as the
> parent
> process is running. When the parent process exits, the tracking
> thread
> terminates and cannot resume when needed.
>
> Currently SSH/SCP/Telnet communication is handled by
> kvm_utils.kvm_spawn, which
> does not allow the child process to run after the parent process
> exits. Thus,
> open SSH/SCP/Telnet sessions cannot be reused by tests following the
> one in
> which they are opened.
>
> The new module provides a solution to these two problems, and also
> saves some
> code by reusing common code required both for QEMU processes and
> SSH/SCP/Telnet
> processes.
>
> Signed-off-by: Michael Goldish <mgoldish@redhat.com>
> ---
>  client/tests/kvm/kvm_subprocess.py | 1146
>  ++++++++++++++++++++++++++++++++++++
>  1 files changed, 1146 insertions(+), 0 deletions(-)
>  create mode 100644 client/tests/kvm/kvm_subprocess.py
>