[PATCH 00/15 ibacm] ACM Cache Preloading (and some cosmetic cleanups)

[PATCH 00/15 ibacm] ACM Cache Preloading (and some cosmetic cleanups)

[Hal Rosenstock]

This patch series has 2 purposes: cleanups of documentation and
configuration files, and cache preloading.

Cache preloading eliminates even the first network query when the
destination is found in the cache. In order to accomplish this,
there are 2 new static configuration files added as options. The
first is an output file for SA PathRecords. The current format
supported is the one produced by the dump_pr plugin to OpenSM.
Other formats can readuly be supported in the future. The second
is a clusters wide file which maps IP address (either IPv4 or
IPv6) to IB GID.

Patches 1-9 clean up documentation and sample config files.
Patch 10 adds the ability to preload the GID and LID destination caches.
Patch 11 updates the ACM documentation for this.
Patch 12 fixes the inet_pton buffer space.
Patch 13 fixes IPv6 support in acme.
Patch 14 adds the ability to preload the IPv4 and IPv6 destination caches.
Patch 15 adds a performance testing option to acme.

Hal Rosenstock (15):
  acm_notes.txt: Change DOS formatting to unix formatting
  ibacm_opts.cfg: Fix commentary typo
  acm.c: Cosmetic formatting change to acm log message
  ibacm*.cfg: Change DOS formatting to unix formatting
  ibacm*.cfg: Fix commentary typos
  ibacm_opts.cfg: Sync default options file with generated output
  More cosmetic changes for consistency between acme and config files
  man: Change DOS formatting to unix formatting
  Update ib_acme.1 and ibacm.1 man pages
  Add the ability to preload the destination GID and LID caches
  Add description of preloading GID and LID destination caches to
    acm_notes.txt
  Increase buffer space for inet_ntop from 32 to INET6_ADDRSTRLEN
  ib_acme.c: Support IPv6 addressing in resolve_ip
  Add support for preloading IPv4 and IPv6 ACM caches
  Add option to allow for ACM cache lookup performance measurement

 Makefile.am        |    2 +-
 acm_notes.txt      |  274 +++++++++++++++++++++-------------------
 ibacm_addr.cfg     |   48 ++++----
 ibacm_hosts.cfg    |   12 ++
 ibacm_opts.cfg     |  282 ++++++++++++++++++++++-------------------
 linux/acme_linux.c |    2 +-
 linux/osd.h        |    3 +
 man/ib_acme.1      |  175 ++++++++++++++------------
 man/ibacm.1        |   41 ++++++-
 man/ibacm.7        |   62 +++++-----
 src/acm.c          |  364 ++++++++++++++++++++++++++++++++++++++++++++++++++--
 src/acme.c         |   96 ++++++++++----
 src/libacm.c       |   20 ++--
 src/libacm.h       |    7 +-
 14 files changed, 937 insertions(+), 451 deletions(-)
 create mode 100644 ibacm_hosts.cfg

-- 
1.7.8.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 00/15 ibacm] ACM Cache Preloading (and some cosmetic cleanups)

[Hefty, Sean]

> Patches 1-9 clean up documentation and sample config files.

Rather than trying to keep the sample ibacm_addr/opts files in sync, I elected to remove them from the git tree.  There's really no use for them, and I don't think they're included with the released package anyway.

Doing this drops patches 2, 4, 5, 6, and part of 7.  I applied 1, 8, 9, and the remainder of 7.  I'll comment on 3 separately.  Patches 10-15 are still pending.

Thanks,
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 00/15 ibacm] ACM Cache Preloading (and some cosmetic cleanups)

[Hefty, Sean]

I've pulled in this series, with the changes mentioned in a separate email, plus updates to the patches below (mostly renames of functions, variables, enums, etc.).  The updated patches are available from my ibacm.git tree in the preload branch.

Please look over those changes and see if you agree with them.  They should be as listed below:

> Patches 1-9 clean up documentation and sample config files.

Removed sample config files from the source tree.

> Patch 10 adds the ability to preload the GID and LID destination caches.

I modified the variable names and enums used for preloading, with other naming changes.

route_preload - is now used to indicate if the routing data should be preloaded and the format to be used.
route_data_file - stores the file name containing the routing data.  The default name is ibacm_route.data

The code to parse the file should be mostly unchanged.

> Patch 11 updates the ACM documentation for this.
> Patch 12 fixes the inet_pton buffer space.
> Patch 13 fixes IPv6 support in acme.

Relocated some common functionality into a static function and used sockaddr_storage to store the resulting addresses.

> Patch 14 adds the ability to preload the IPv4 and IPv6 destination caches.

Name changes similar to those done for patch 10.

addr_preload - indicates if the address data should be preloaded and the format to use
addr_data_file - stores the file name containing the address data.  The default name is ibacm_addr.data.

> Patch 15 adds a performance testing option to acme.

Overall, the changes look good.  I just needed to figure out what to call everything and tried to keep things consistent.  My preload branch has been compile tested, but I have not yet run it.

I only noticed one gap in the implementation, which I commented on directly in the code.  The changes require that the routing data be loaded first before the address data.  This is the opposite of what occurs for normal operation, where the address is resolved before the route.  It would be nice if the address and routing preload data were more independent, but that can be added later, maybe, someday.

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 00/15 ibacm] ACM Cache Preloading (and some cosmetic cleanups)

[Hal Rosenstock]

On 6/28/2013 3:45 AM, Hefty, Sean wrote:
> I've pulled in this series, with the changes mentioned in a separate email, plus updates to the patches below (mostly renames of functions, variables, enums, etc.).  The updated patches are available from my ibacm.git tree in the preload branch.

Thanks for getting this done so quickly!

> Please look over those changes and see if you agree with them.  

I looked them over and retested this. I have a few minor fixups to
follow shortly.

> They should be as listed below:
> 
>> Patches 1-9 clean up documentation and sample config files.
> 
> Removed sample config files from the source tree.
> 
>> Patch 10 adds the ability to preload the GID and LID destination caches.
> 
> I modified the variable names and enums used for preloading, with other naming changes.
> 
> route_preload - is now used to indicate if the routing data should be preloaded and the format to be used.
> route_data_file - stores the file name containing the routing data.  The default name is ibacm_route.data
> 
> The code to parse the file should be mostly unchanged.

I didn't notice any parsing changes but might have glanced over this.

>> Patch 11 updates the ACM documentation for this.
>> Patch 12 fixes the inet_pton buffer space.
>> Patch 13 fixes IPv6 support in acme.
> 
> Relocated some common functionality into a static function and used sockaddr_storage to store the resulting addresses.
> 
>> Patch 14 adds the ability to preload the IPv4 and IPv6 destination caches.
> 
> Name changes similar to those done for patch 10.
> 
> addr_preload - indicates if the address data should be preloaded and the format to use
> addr_data_file - stores the file name containing the address data.  The default name is ibacm_addr.data.

Nit: Default is ibacm_hosts.data rather than ibacm_addr.data.

The one thing I noticed is that the check for using this (address)
preload option without the previous (route) preload option was
eliminated. This is probably the case in the long run but it is not the
case with these current options. Should it be added back in ?

>> Patch 15 adds a performance testing option to acme.
> 
> Overall, the changes look good.  I just needed to figure out what to call everything and tried to keep things consistent.  My preload branch has been compile tested, but I have not yet run it.
> 
> I only noticed one gap in the implementation, which I commented on directly in the code. 
> The changes require that the routing data be loaded first before the address data.  This is 
> the opposite of what occurs for normal operation, where the address is resolved before the route.

Understood. To restate this, it stems from address preload being host/IP
to IB GID and route preload being GID and LID to PR so the later is
needed first to populate the former (to PRs).

> It would be nice if the address and routing preload data were more independent, but that can be 
> added later, maybe, someday.

Agreed. I'll keep this in mind in terms of future changes.

-- Hal

> - Sean
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 00/15 ibacm] ACM Cache Preloading (and some cosmetic cleanups)

[Hefty, Sean]

> I looked them over and retested this. I have a few minor fixups to
> follow shortly.

I've incorporated your changes into the patch set and updated the preload branch.
 
> The one thing I noticed is that the check for using this (address)
> preload option without the previous (route) preload option was
> eliminated. This is probably the case in the long run but it is not the
> case with these current options. Should it be added back in ?

The check is there, but no error is generated.  It may not be that difficult to avoid this dependency, but I'll respond more directly to affected patch.

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html