[PATCH 1/2] cooker.py: Close lock file and watcher fds when end.

[PATCH 1/2] cooker.py: Close lock file and watcher fds when end.

[Aníbal Limón]

When run cooker inside loop it reaches OS max fd's causing
an exception.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 lib/bb/cooker.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index f31bca6..abe8a52 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -1432,6 +1432,9 @@ class BBCooker:
         if self.state in (state.shutdown, state.forceshutdown, state.error):
             if hasattr(self.parser, 'shutdown'):
                 self.parser.shutdown(clean=False, force = True)
+            self.configwatcher.close()
+            self.watcher.close()
+            self.lock.close()
             raise bb.BBHandledException()
 
         if self.state != state.parsing:
-- 
1.9.1

[PATCH 2/2] tinfoil.py: Add shutdown method.

[Aníbal Limón]

When run tinfoil in a loop it causing problems for don't
free resources like OS max fd exception.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 lib/bb/tinfoil.py | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/lib/bb/tinfoil.py b/lib/bb/tinfoil.py
index 8fc9be3..5025e62 100644
--- a/lib/bb/tinfoil.py
+++ b/lib/bb/tinfoil.py
@@ -28,6 +28,7 @@ import bb.utils
 from bb.cooker import state, BBCooker, CookerFeatures
 from bb.cookerdata import CookerConfiguration, ConfigParameters
 import bb.fetch2
+from bb import BBHandledException
 
 class Tinfoil:
     def __init__(self, output=sys.stdout, tracking=False):
@@ -84,6 +85,13 @@ class Tinfoil:
             else:
                 self.parseRecipes()
 
+    def shutdown(self):
+        try:
+            self.cooker.shutdown()
+            self.cooker.updateCache()
+        except BBHandledException:
+            pass
+
 class TinfoilConfigParameters(ConfigParameters):
 
     def __init__(self, **options):
-- 
1.9.1

Re: [PATCH 1/2] cooker.py: Close lock file and watcher fds when end.

[Richard Purdie]

On Fri, 2015-07-03 at 15:40 -0500, Aníbal Limón wrote:
> When run cooker inside loop it reaches OS max fd's causing
> an exception.
> 
> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
> ---
>  lib/bb/cooker.py | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
> index f31bca6..abe8a52 100644
> --- a/lib/bb/cooker.py
> +++ b/lib/bb/cooker.py
> @@ -1432,6 +1432,9 @@ class BBCooker:
>          if self.state in (state.shutdown, state.forceshutdown, state.error):
>              if hasattr(self.parser, 'shutdown'):
>                  self.parser.shutdown(clean=False, force = True)
> +            self.configwatcher.close()
> +            self.watcher.close()
> +            self.lock.close()
>              raise bb.BBHandledException()
>  
>          if self.state != state.parsing:

Burying this cleanup into the updateCache function seems wrong. The
updateCache call as you use it in tinfoil highlights how odd that is.

We really need a better shutdown mechanism of cooker. It'll be more
invasive to the codebase but better in the long run I think...

Cheers,

Richard