[PATCH] imsm: Forbid spanning between multiple controllers.

[PATCH] imsm: Forbid spanning between multiple controllers.

[Marcin Tomczak]

Attaching disks to multiple controllers of the same type has been
allowed so far. Now spanning between multiple controllers is disallowed
at all by IMSM metadata.

Signed-off-by: Marcin Tomczak <marcin.tomczak@intel.com>
Reviewed-by: Lukasz Dorau <lukasz.dorau@intel.com>
---
 super-intel.c |   25 ++++++++-----------------
 1 files changed, 8 insertions(+), 17 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 544cb05..db437b9 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -558,20 +558,11 @@ static int attach_hba_to_super(struct intel_super *super, struct sys_dev *device
 	if (super->hba == NULL) {
 		super->hba = alloc_intel_hba(device);
 		return 1;
-	}
-
-	hba = super->hba;
-	/* Intel metadata allows for all disks attached to the same type HBA.
-	 * Do not sypport odf HBA types mixing
-	 */
-	if (device->type != hba->type)
+	} else
+		/* IMSM metadata disallows to attach disks to multiple
+		 * controllers.
+		 */
 		return 2;
-
-	while (hba->next)
-		hba = hba->next;
-
-	hba->next = alloc_intel_hba(device);
-	return 1;
 }
 
 static struct sys_dev* find_disk_attached_hba(int fd, const char *devname)
@@ -3018,11 +3009,11 @@ static int compare_super_imsm(struct supertype *st, struct supertype *tst)
 	 */
 	if (!check_env("IMSM_NO_PLATFORM")) {
 		if (!first->hba || !sec->hba ||
-		    (first->hba->type != sec->hba->type))  {
+		    strcmp(first->hba->path, sec->hba->path) != 0)  {
 			fprintf(stderr,
 				"HBAs of devices does not match %s != %s\n",
-				first->hba ? get_sys_dev_type(first->hba->type) : NULL,
-				sec->hba ? get_sys_dev_type(sec->hba->type) : NULL);
+				first->hba ? first->hba->path : NULL,
+				sec->hba ? sec->hba->path : NULL);
 			return 3;
 		}
 	}
@@ -3800,7 +3791,7 @@ static int find_intel_hba_capability(int fd, struct intel_super *super, char *de
 			}
 
 			fprintf(stderr, ").\n"
-				"    Mixing devices attached to different controllers "
+				"    Mixing devices attached to multiple controllers "
 				"is not allowed.\n");
 		}
 		free_sys_dev(&hba_name);

---------------------------------------------------------------------
Intel Technology Poland sp. z o.o.
z siedziba w Gdansku
ul. Slowackiego 173
80-298 Gdansk

Sad Rejonowy Gdansk Polnoc w Gdansku, 
VII Wydzial Gospodarczy Krajowego Rejestru Sadowego, 
numer KRS 101882

NIP 957-07-52-316
Kapital zakladowy 200.000 zl

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [PATCH] imsm: Forbid spanning between multiple controllers.

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2349 bytes --]

On Fri, 09 Nov 2012 15:46:36 +0100 Marcin Tomczak <marcin.tomczak@intel.com>
wrote:

> Attaching disks to multiple controllers of the same type has been
> allowed so far. Now spanning between multiple controllers is disallowed
> at all by IMSM metadata.
> 
> Signed-off-by: Marcin Tomczak <marcin.tomczak@intel.com>
> Reviewed-by: Lukasz Dorau <lukasz.dorau@intel.com>
> ---
>  super-intel.c |   25 ++++++++-----------------
>  1 files changed, 8 insertions(+), 17 deletions(-)
> 
> diff --git a/super-intel.c b/super-intel.c
> index 544cb05..db437b9 100644
> --- a/super-intel.c
> +++ b/super-intel.c
> @@ -558,20 +558,11 @@ static int attach_hba_to_super(struct intel_super *super, struct sys_dev *device
>  	if (super->hba == NULL) {
>  		super->hba = alloc_intel_hba(device);
>  		return 1;
> -	}
> -
> -	hba = super->hba;
> -	/* Intel metadata allows for all disks attached to the same type HBA.
> -	 * Do not sypport odf HBA types mixing
> -	 */
> -	if (device->type != hba->type)
> +	} else
> +		/* IMSM metadata disallows to attach disks to multiple
> +		 * controllers.
> +		 */
>  		return 2;
> -
> -	while (hba->next)
> -		hba = hba->next;
> -
> -	hba->next = alloc_intel_hba(device);
> -	return 1;
>  }
>  
>  static struct sys_dev* find_disk_attached_hba(int fd, const char *devname)
> @@ -3018,11 +3009,11 @@ static int compare_super_imsm(struct supertype *st, struct supertype *tst)
>  	 */
>  	if (!check_env("IMSM_NO_PLATFORM")) {
>  		if (!first->hba || !sec->hba ||
> -		    (first->hba->type != sec->hba->type))  {
> +		    strcmp(first->hba->path, sec->hba->path) != 0)  {
>  			fprintf(stderr,
>  				"HBAs of devices does not match %s != %s\n",
> -				first->hba ? get_sys_dev_type(first->hba->type) : NULL,
> -				sec->hba ? get_sys_dev_type(sec->hba->type) : NULL);
> +				first->hba ? first->hba->path : NULL,
> +				sec->hba ? sec->hba->path : NULL);
>  			return 3;
>  		}
>  	}
> @@ -3800,7 +3791,7 @@ static int find_intel_hba_capability(int fd, struct intel_super *super, char *de
>  			}
>  
>  			fprintf(stderr, ").\n"
> -				"    Mixing devices attached to different controllers "
> +				"    Mixing devices attached to multiple controllers "
>  				"is not allowed.\n");
>  		}
>  		free_sys_dev(&hba_name);
> 

applied, thanks.

NeilBrown


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: [PATCH] imsm: Forbid spanning between multiple controllers.

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/9/2012 9:46 AM, Marcin Tomczak wrote:
> Attaching disks to multiple controllers of the same type has been 
> allowed so far. Now spanning between multiple controllers is
> disallowed at all by IMSM metadata.

Does this mean that if you have a failure and try to recover by
plugging the disks into another machine where they are possibly not on
an IMSM controller at all, or split across different controllers,
mdadm will refuse to assemble the array?



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iQEcBAEBAgAGBQJQ7zlhAAoJEJrBOlT6nu75n/MIANZdgRyKBvJbKdLJlPtxzLLY
b1wlizB29YpX/tP+fJ7/O5r7u83tH/xFfruhSHgPSjnWxae8DGH84CLrs0uvMF+y
KD4/wacM288pWhq5WLLg9GeP266UAFAPNL3no39J6cZdOvPCLSrjsuvIx6YmSZ6E
2JniYreReDGoGqGvaLRb/dT2Qx6vF5D8M/6wV0ghuHlUYGx21QeMdHmI4SQmn8tJ
NGDl9p3bAYZ7poRhkTczOyICIhB33hg/hYkMoMc9I64U03kner4LNxsxE05zICZo
C/94KDWbIJV6Le6kxEE0pxeSKSrLj0I8kv/HoFUAi9cJY25y1N1IWQwqD3E35lg=
=GSbm
-----END PGP SIGNATURE-----

Re: [PATCH] imsm: Forbid spanning between multiple controllers.

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 1/14/2013 8:02 AM, Tomczak, Marcin wrote:
> Hi. Yes, mdadm will refuse any commands, when you try to use disks
> attached to multiple controllers. Also mdadm supports IMSM metadata
> only if you have a machine with Intel OROM (and the patch "Forbid
> spanning" changes nothing about it).

Why is this?  One of the nice things about dmraid was that since it
really was just software raid, if your controller blew you could
recover your data by moving the drives to another machine and still
recognize them.  Why should mdadm care about the controller?

I suppose it is nice to warn people that they are trying to create an
array that the bios won't recognize, but mdadm should not refuse to
run an array just because some hardware that is completely unnecessary
is not found.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJQ9BnyAAoJEJrBOlT6nu75tikH/jQAsx4waselA6rrKC8aFD7+
oG5pphTskS10iukkJGX8d+o/HiiMM66IuMqM3vRSNZO40P6arhAETE02jvNY9vFN
fI61AMpCv0FvBFv83RRmh19HkHLQEgiztcLRI7GeTNl5aOnC4wk8NY/Rqt022bhV
MGHickXUDNXtoWSXE3x03aXm9cwjIwfPbDrt3jOyrf9EqgZZW2j8uqY3PbH5Zcav
slKjFOaVt98nvLeDLpBJ6pR4vKq3Qax+pOPNdvlGG0GAqftH/iEYHsopmM4D4o5L
21cI1CRfmyaF8ro0ryZPJkQVe46Ya41Qb1wd9Y6pg5mfbqGiwbQvMHi7o53Mc70=
=NumJ
-----END PGP SIGNATURE-----

RE: [PATCH] imsm: Forbid spanning between multiple controllers.

[Patelczyk, Maciej]

> -----Original Message-----
> From: Phillip Susi [mailto:psusi@ubuntu.com]
> Sent: Monday, January 14, 2013 3:45 PM
> To: Tomczak, Marcin
> Cc: neilb@suse.de; linux-raid@vger.kernel.org; Patelczyk, Maciej;
> Dorau, Lukasz
> Subject: Re: [PATCH] imsm: Forbid spanning between multiple
> controllers.
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 1/14/2013 8:02 AM, Tomczak, Marcin wrote:
> > Hi. Yes, mdadm will refuse any commands, when you try to use disks
> > attached to multiple controllers. Also mdadm supports IMSM metadata
> > only if you have a machine with Intel OROM (and the patch "Forbid
> > spanning" changes nothing about it).
> 
> Why is this?  One of the nice things about dmraid was that since it
> really was just software raid, if your controller blew you could
> recover your data by moving the drives to another machine and still
> recognize them.  Why should mdadm care about the controller?
> 
> I suppose it is nice to warn people that they are trying to create an
> array that the bios won't recognize, but mdadm should not refuse to run
> an array just because some hardware that is completely unnecessary is
> not found.
> 
> 

Hi Phillip,

Mdadm does not care about the controller unless you created IMSM based RAID. Basically you can create that type of RAID *only* on Intel based platforms with OROM enabled. It's Intel solution, we support it and we maintain it. It's very specific type of metadata.
We provide this functionality in Linux, Windows and OROM/uEFI. It must be compatible between all three environments. If you have dual boot machine you can safely boot into Windows and then into Linux and your RAID Volume will be in proper state.

Dmraid cannot boot from RAID Volume, OROM can boot from IMSM RAID Volume directly. You don't need separate partition, hard drive or anything else. If you run Intel platform you can boot directly from supported RAID Volume. This is because OROM supports it. Mdadm respects OROM restrictions when creating IMSM based RAIDs.

Spanning in Linux is something obvious. I know that is simply works because of Linux architecture. However spanned RAID Volumes are not supported in OROM and in Windows. If you allowed for spanned RAID Volumes in Linux we open the Pandora's box. In the worst case you will lose you data when entering to OROM (OROM will see only one set of disks attached to one controller and can mark RAID Volume as Failed) or if you boot to Windows (Windows driver will see two failed RAID Volumes in the worst case). In other case you will have RAID Volume marked as Degraded and rebuild will start. And what if you create 'bootable' RAID Volume? Well you may not be able to boot from it when it's spanned.

It's all about compatibility. Windows and OROM don't support spanned RAID Volumes so must Linux.

Note that mdadm is not limited to IMSM metadata. You can use DDF or native metadata. It's just an option. But if you use IMSM metadata you are limited to Intel platform.

I hope that I clarified a little bit.
maciej




> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (MingW32)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
> 
> iQEcBAEBAgAGBQJQ9BnyAAoJEJrBOlT6nu75tikH/jQAsx4waselA6rrKC8aFD7+
> oG5pphTskS10iukkJGX8d+o/HiiMM66IuMqM3vRSNZO40P6arhAETE02jvNY9vFN
> fI61AMpCv0FvBFv83RRmh19HkHLQEgiztcLRI7GeTNl5aOnC4wk8NY/Rqt022bhV
> MGHickXUDNXtoWSXE3x03aXm9cwjIwfPbDrt3jOyrf9EqgZZW2j8uqY3PbH5Zcav
> slKjFOaVt98nvLeDLpBJ6pR4vKq3Qax+pOPNdvlGG0GAqftH/iEYHsopmM4D4o5L
> 21cI1CRfmyaF8ro0ryZPJkQVe46Ya41Qb1wd9Y6pg5mfbqGiwbQvMHi7o53Mc70=
> =NumJ
> -----END PGP SIGNATURE-----

Re: [PATCH] imsm: Forbid spanning between multiple controllers.

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 1/15/2013 5:31 AM, Patelczyk, Maciej wrote:
> Hi Phillip,
> 
> Mdadm does not care about the controller unless you created IMSM 
> based RAID. Basically you can create that type of RAID *only* on 
> Intel based platforms with OROM enabled. It's Intel solution, we 
> support it and we maintain it. It's very specific type of metadata.
> We provide this functionality in Linux, Windows and OROM/uEFI. It
> must be compatible between all three environments. If you have dual
> boot machine you can safely boot into Windows and then into Linux
> and your RAID Volume will be in proper state.

> Dmraid cannot boot from RAID Volume, OROM can boot from IMSM RAID 
> Volume directly. You don't need separate partition, hard drive or 
> anything else. If you run Intel platform you can boot directly
> from supported RAID Volume. This is because OROM supports it.
> Mdadm respects OROM restrictions when creating IMSM based RAIDs.

I understand that you will not be able to boot directly from the array
without the OROM, but sometimes you just need to access your data any
way you can, and mdadm should not refuse to mount the array based on
the absence of the OROM.  Think disaster recovery.  You should always
be able to connect the disks to another machine that may not have an
Intel controller and rescue your data.

Additionally the ability to create and use arrays without the OROM
allows for simulation and testing, which is something the mdadm test
suite should be doing.

> Spanning in Linux is something obvious. I know that is simply
> works because of Linux architecture. However spanned RAID Volumes
> are not supported in OROM and in Windows. If you allowed for
> spanned RAID Volumes in Linux we open the Pandora's box. In the
> worst case you will lose you data when entering to OROM (OROM will
> see only one set of disks attached to one controller and can mark
> RAID Volume as Failed) or if you boot to Windows (Windows driver
> will see two failed RAID Volumes in the worst case). In other case
> you will have RAID Volume marked as Degraded and rebuild will
> start. And what if you create 'bootable' RAID Volume? Well you may
> not be able to boot from it when it's spanned.

Warnings about potentially troublesome situations are good, but
outright refusal is not.  Yes, I realize it would be a problem for
Windows due to the poor way the driver has to be implemented ( why
can't the OROM see other disks on other controllers? ), but sometimes
you don't care about that.  For instance, if you are setting up the
array on one machine where you can not connect all of the drives to
the same controller ( and do not care about booting from the array on
this machine ), but you are planning on moving them to a machine where
they will be.  This is just one example of many situations where you
need to be able to say "I know what I'm doing, go ahead anyway".

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJQ9W5SAAoJEJrBOlT6nu75mZIH/iusNcvgb+Gu2CWmjeSjfcNA
b66NMNdN63csAew9chaya71MxbET9RE1+SQyTXQZkz6tbtbrxP/H2gu0JzNLjO3T
G2PsXo1edHD9YTkx7wUImJgCANglqWmvbDwDtR5EFuyoEytzFp7PJCzcbPJM8DRU
c60j8OWOmxtbkCnYyQmmOPJD3zB9t9UdFnYdzr+ACnj9+3j0Z950gehw7TaOEKMh
1jTwR9SftdH979GHL2EsLQciyFLPfcT8/Xev1UnYWyj8DSmFs591fNI1zI+t/EsY
anBTJeoD0hkbqe67TaqZSnpX5acEqP1xHmCeAfECfmkreH7JTPwjMQOv457+Na8=
=eKzx
-----END PGP SIGNATURE-----

Re: [PATCH] imsm: Forbid spanning between multiple controllers.

[John Robinson]

On 15/01/2013 14:57, Phillip Susi wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 1/15/2013 5:31 AM, Patelczyk, Maciej wrote:
>> Hi Phillip,
>>
>> Mdadm does not care about the controller unless you created IMSM
>> based RAID. Basically you can create that type of RAID *only* on
>> Intel based platforms with OROM enabled. It's Intel solution, we
>> support it and we maintain it. It's very specific type of metadata.
[...]
>
> Warnings about potentially troublesome situations are good, but
> outright refusal is not.  Yes, I realize it would be a problem for
> Windows due to the poor way the driver has to be implemented ( why
> can't the OROM see other disks on other controllers? ), but sometimes
> you don't care about that.  For instance, if you are setting up the
> array on one machine where you can not connect all of the drives to
> the same controller ( and do not care about booting from the array on
> this machine ), but you are planning on moving them to a machine where
> they will be.  This is just one example of many situations where you
> need to be able to say "I know what I'm doing, go ahead anyway".

Isn't there, or wasn't there, an environment variable for mdadm called 
something like IMSM_PLATFORM_IGNORE for precisely this scenario? Is that 
what this patch series disabled?

Cheers,

John.

RE: [PATCH] imsm: Forbid spanning between multiple controllers.

[Patelczyk, Maciej]

> -----Original Message-----
> From: Phillip Susi [mailto:psusi@ubuntu.com]
> Sent: Tuesday, January 15, 2013 3:57 PM
> To: Patelczyk, Maciej
> Cc: Tomczak, Marcin; neilb@suse.de; linux-raid@vger.kernel.org; Dorau,
> Lukasz
> Subject: Re: [PATCH] imsm: Forbid spanning between multiple
> controllers.
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 1/15/2013 5:31 AM, Patelczyk, Maciej wrote:
> > Hi Phillip,
> >
> > Mdadm does not care about the controller unless you created IMSM
> > based RAID. Basically you can create that type of RAID *only* on
> > Intel based platforms with OROM enabled. It's Intel solution, we
> > support it and we maintain it. It's very specific type of metadata.
> > We provide this functionality in Linux, Windows and OROM/uEFI. It
> > must be compatible between all three environments. If you have dual
> > boot machine you can safely boot into Windows and then into Linux
> > and your RAID Volume will be in proper state.
> 
> > Dmraid cannot boot from RAID Volume, OROM can boot from IMSM RAID
> > Volume directly. You don't need separate partition, hard drive or
> > anything else. If you run Intel platform you can boot directly
> > from supported RAID Volume. This is because OROM supports it.
> > Mdadm respects OROM restrictions when creating IMSM based RAIDs.
> 
> I understand that you will not be able to boot directly from the array
> without the OROM, but sometimes you just need to access your data any
> way you can, and mdadm should not refuse to mount the array based on
> the absence of the OROM.  Think disaster recovery.  You should always
> be able to connect the disks to another machine that may not have an
> Intel controller and rescue your data.
> 
> Additionally the ability to create and use arrays without the OROM
> allows for simulation and testing, which is something the mdadm test
> suite should be doing.
> 

I agree. That is why you have IMSM_NO_PLATFORM flag.
Just do:
$export IMSM_NO_PLATFORM=1
$mdadm ...

And it should work as you mentioned.
Right now this option is described in manual.



> > Spanning in Linux is something obvious. I know that is simply
> > works because of Linux architecture. However spanned RAID Volumes
> > are not supported in OROM and in Windows. If you allowed for
> > spanned RAID Volumes in Linux we open the Pandora's box. In the
> > worst case you will lose you data when entering to OROM (OROM will
> > see only one set of disks attached to one controller and can mark
> > RAID Volume as Failed) or if you boot to Windows (Windows driver
> > will see two failed RAID Volumes in the worst case). In other case
> > you will have RAID Volume marked as Degraded and rebuild will
> > start. And what if you create 'bootable' RAID Volume? Well you may
> > not be able to boot from it when it's spanned.
> 
> Warnings about potentially troublesome situations are good, but
> outright refusal is not.  Yes, I realize it would be a problem for
> Windows due to the poor way the driver has to be implemented ( why
> can't the OROM see other disks on other controllers? ), but sometimes
> you don't care about that.  For instance, if you are setting up the
> array on one machine where you can not connect all of the drives to
> the same controller ( and do not care about booting from the array on
> this machine ), but you are planning on moving them to a machine where
> they will be.  This is just one example of many situations where you
> need to be able to say "I know what I'm doing, go ahead anyway".
> 

Of course I agree in general. This is standard Unix way to do it.
I'm root and I take full responsibility. I think that IMSM_NO_PLATFORM
should handle spanning too. I will discuss it internally.

Errr..., "I know what I'm doing, go ahead anyway" - well not quite so...
Do you really know the target platform OROM's capabilities? Do you?

When creating or assembling IMSM RAID mdadm looks for OROM and tries to 
determine its capabilities. You can look at the code (like platform-intel.h).
When you create RAID Volume directly on target machine you're safe. Mdadm
will prevent you from creating OROM incompatible RAID. If you create the
RAID Volume with IMSM_NO_PLAFTORM flag or with different OROM you may be
in situation is which some capabilities in one OROM will not be present in
other OROM and then RAID Volume will not be assembled. Target OROM may not
support given RAID Level, Strip Size or some other capability. Yes I know that is
not nice, but this is how it works. There are many platforms, some changed by
Vendors and we don't control all of it. If you create RAID with IMSM_NO_PLATFORM
it's even worse ;-)
So it's not so obvious with this 'go ahead' method.
Of course you can always run mdadm --detail-platform on target platform and check
if you're creating OROM compatible RAID Volume. 


maciej

RE: [PATCH] imsm: Forbid spanning between multiple controllers.

[Patelczyk, Maciej]

> I agree. That is why you have IMSM_NO_PLATFORM flag.
> Just do:
> $export IMSM_NO_PLATFORM=1
> $mdadm ...
> 
> And it should work as you mentioned.
> Right now this option is described in manual.
                     ^^^^^^^^^^
Is *not* that is.

Re: [PATCH] imsm: Forbid spanning between multiple controllers.

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 1/16/2013 4:58 AM, Patelczyk, Maciej wrote:
> I agree. That is why you have IMSM_NO_PLATFORM flag. Just do: 
> $export IMSM_NO_PLATFORM=1 $mdadm ...

That should do, and a man page entry would be nice ;)

> Of course I agree in general. This is standard Unix way to do it. 
> I'm root and I take full responsibility. I think that
> IMSM_NO_PLATFORM should handle spanning too. I will discuss it
> internally.
> 
> Errr..., "I know what I'm doing, go ahead anyway" - well not quite
> so... Do you really know the target platform OROM's capabilities?
> Do you?

Maybe I do, maybe I don't, but as you said, I'm root and I take full
responsibility.  You did me a favor by warning me, now let me take the
safety off and if I shoot myself in the foot, so be it.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJQ9riJAAoJEJrBOlT6nu75hVQH/0AJjr6yLe8Dts103npPSjPZ
D3N9TaloCiBVVsn9Txl7Ws4oOeV771by0PyK858jMNZR47tvlRDcR4x1xQPV7o3G
TtOFLLTl+3HDdXzUz9QTJA5xpggEt3AskKFz/FZCvbSFt9btqX7/MOPApY8K2ulY
vIsqBmGnw61RsgvzjXsWIqSqouzXFBb5pFOqKBpstUWWc1H2Ug0rJuTJjilLxtGy
rcxPuBgBZcn38bdRmzYB3qHhVPmz+k6bndCjQbmefgq9FS5J5jKFlA2J3Djt5LVy
LONGSKog1+y42X8f3mozpK47xSX1vdgU5ef3U5RANzWzC52yPST+XAxzBjj+z/s=
=FXt6
-----END PGP SIGNATURE-----