From mboxrd@z Thu Jan 1 00:00:00 1970 From: lhh@sourceware.org Date: 13 Nov 2006 16:14:10 -0000 Subject: [Cluster-devel] cluster fence/agents/xvm/fence_xvm.c fence/age ... Message-ID: <20061113161410.2846.qmail@sourceware.org> List-Id: To: cluster-devel.redhat.com MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit CVSROOT: /cvs/cluster Module name: cluster Branch: RHEL5 Changes by: lhh at sourceware.org 2006-11-13 16:14:08 Modified files: fence/agents/xvm: fence_xvm.c options.h xvm.h Makefile TODO options.c vm_states.c tcp.c fence_xvmd.c mcast.c ip_lookup.c simple_auth.c cman/init.d : cman fence/man : fence_xvmd.8 fence_xvm.8 Added files: fence/agents/xvm: options-ccs.c debug.c Log message: Fix bugzilla #212474; fully integrates fence_xvmd with ccs & the cman init script Patches: http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/options-ccs.c.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=NONE&r2=1.1.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/debug.c.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=NONE&r2=1.1.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/fence_xvm.c.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.3&r2=1.3.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/options.h.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.1&r2=1.1.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/xvm.h.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.1&r2=1.1.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/Makefile.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.3&r2=1.3.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/TODO.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.1&r2=1.1.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/options.c.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.2&r2=1.2.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/vm_states.c.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.1&r2=1.1.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/tcp.c.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.1&r2=1.1.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/fence_xvmd.c.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.4&r2=1.4.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/mcast.c.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.1&r2=1.1.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/ip_lookup.c.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.2&r2=1.2.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/agents/xvm/simple_auth.c.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.1&r2=1.1.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/cman/init.d/cman.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.26&r2=1.26.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/man/fence_xvmd.8.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.1&r2=1.1.2.1 http://sourceware.org/cgi-bin/cvsweb.cgi/cluster/fence/man/fence_xvm.8.diff?cvsroot=cluster&only_with_tag=RHEL5&r1=1.1&r2=1.1.2.1 /cvs/cluster/cluster/fence/agents/xvm/options-ccs.c,v --> standard output revision 1.1.2.1 --- cluster/fence/agents/xvm/options-ccs.c +++ - 2006-11-13 16:14:08.497837000 +0000 @@ -0,0 +1,115 @@ +/* + Copyright Red Hat, Inc. 2006 + + This program is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by the + Free Software Foundation; either version 2, or (at your option) any + later version. + + This program is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; see the file COPYING. If not, write to the + Free Software Foundation, Inc., 675 Mass Ave, Cambridge, + MA 02139, USA. +*/ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* Local includes */ +#include "xvm.h" +#include "simple_auth.h" +#include "mcast.h" +#include "options.h" + +#include + +struct arg_info *find_arg_by_char(char arg); +struct arg_info *find_arg_by_string(char *arg); + +extern int _debug; + +/** + Parse args from ccs and assign to the specified args structure. + (This should only be called from fence_xvmd; not fence_xvm!!!) + + @param optstr Command line option string in getopt(3) format + @param args Args structure to fill in. + */ +void +args_get_ccs(char *optstr, fence_xvm_args_t *args) +{ + char buf[256]; + int ccsfd = -1, x, n; + char *val; + struct arg_info *arg; + + if (args->flags & (F_NOCCS | F_HELP | F_VERSION)) + return; + + ccsfd = ccs_connect(); + if (ccsfd < 0) { + args->flags |= F_CCSFAIL; + return; + } + + for (x = 0; x < strlen(optstr); x++) { + arg = find_arg_by_char(optstr[x]); + if (!arg) + continue; + + if (!arg || (arg->opt != '\xff' && + !strchr(optstr, arg->opt))) { + continue; + } + + n = snprintf(buf, sizeof(buf), "/cluster/fence_xvmd/@%s\n", + arg->stdin_opt); + if (n == sizeof(buf)) { + args->flags |= F_CCSERR; + return; + } + + val = NULL; + if (ccs_get(ccsfd, buf, &val) != 0) { + if (val) { + free(val); + val = NULL; + } + continue; + } + + if (!val) + continue; + + if (arg->assign) + arg->assign(args, arg, val); + + if (val) { + free(val); + val = NULL; + } + } + + ccs_disconnect(ccsfd); +} /cvs/cluster/cluster/fence/agents/xvm/debug.c,v --> standard output revision 1.1.2.1 --- cluster/fence/agents/xvm/debug.c +++ - 2006-11-13 16:14:08.583897000 +0000 @@ -0,0 +1,34 @@ +/* + Copyright Red Hat, Inc. 2006 + + This program is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by the + Free Software Foundation; either version 2, or (at your option) any + later version. + + This program is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; see the file COPYING. If not, write to the + Free Software Foundation, Inc., 675 Mass Ave, Cambridge, + MA 02139, USA. +*/ +#include "xvm.h" + +static int _debug = 0; + +inline void +dset(int threshold) +{ + _debug = threshold; + dprintf(3, "Debugging threshold is now %d\n", threshold); +} + +inline int +dget(void) +{ + return _debug; +} --- cluster/fence/agents/xvm/fence_xvm.c 2006/11/03 15:58:56 1.3 +++ cluster/fence/agents/xvm/fence_xvm.c 2006/11/13 16:14:06 1.3.2.1 @@ -63,6 +63,7 @@ int n; struct timeval tv; + dprintf(3, "Waiting for connection from XVM host daemon.\n"); FD_ZERO(&rfds); FD_SET(lfd, &rfds); tv.tv_sec = retry_tenths / 10; @@ -93,6 +94,7 @@ struct timeval tv; /* Ok, we're connected */ + dprintf(3, "Issuing TCP challenge\n"); if (tcp_challenge(fd, auth, key, key_len, timeout) <= 0) { /* Challenge failed */ printf("Invalid response to challenge\n"); @@ -100,12 +102,13 @@ } /* Now they'll send us one, so we need to respond here */ + dprintf(3, "Responding to TCP challenge\n"); if (tcp_response(fd, auth, key, key_len, timeout) <= 0) { printf("Invalid response to challenge\n"); return 0; } - printf("TCP Exchange + Authentication done... \n"); + dprintf(2, "TCP Exchange + Authentication done... \n"); FD_ZERO(&rfds); FD_SET(fd, &rfds); @@ -113,6 +116,7 @@ tv.tv_usec = 0; ret = 1; + dprintf(3, "Waiting for return value from XVM host\n"); if (select(fd + 1, &rfds, NULL, NULL, &tv) <= 0) return -1; @@ -142,7 +146,7 @@ for (ipa = ipl->tqh_first; ipa; ipa = ipa->ipa_entries.tqe_next) { if (ipa->ipa_family != args->family) { - printf("Ignoring %s: wrong family\n", ipa->ipa_address); + dprintf(2, "Ignoring %s: wrong family\n", ipa->ipa_address); continue; } @@ -162,7 +166,7 @@ tgt = (struct sockaddr *)&tgt6; tgt_len = sizeof(tgt6); } else { - printf("Unsupported family %d\n", args->family); + dprintf(2, "Unsupported family %d\n", args->family); return -1; } @@ -194,8 +198,8 @@ sign_request(&freq, key, key_len); - printf("Sending to %s via %s\n", args->addr, - ipa->ipa_address); + dprintf(3, "Sending to %s via %s\n", args->addr, + ipa->ipa_address); sendto(mc_sock, &freq, sizeof(freq), 0, (struct sockaddr *)tgt, tgt_len); @@ -213,13 +217,15 @@ { ip_list_t ipl; char key[4096]; - int lfd, key_len, fd; + int lfd, key_len = 0, fd; int attempts = 0; - - key_len = read_key_file(args->key_file, key, sizeof(key)); - if (key_len < 0) { - printf("Key file unreadable!\n"); - return 1; + + if (args->auth != AUTH_NONE || args->hash != HASH_NONE) { + key_len = read_key_file(args->key_file, key, sizeof(key)); + if (key_len < 0) { + printf("Could not read key file\n"); + return 1; + } } /* Do the real work */ @@ -246,7 +252,7 @@ } if (lfd < 0) { - printf("failed to listen: %s\n", strerror(errno)); + printf("Failed to listen: %s\n", strerror(errno)); return 1; } @@ -332,8 +338,9 @@ } args_finalize(&args); - - if (args.flags & F_DEBUG) + dset(args.debug); + + if (args.debug > 0) args_print(&args); /* Additional validation here */ --- cluster/fence/agents/xvm/options.h 2006/10/05 16:11:36 1.1 +++ cluster/fence/agents/xvm/options.h 2006/11/13 16:14:06 1.1.2.1 @@ -19,6 +19,18 @@ #ifndef _XVM_OPTIONS_H #define _XVM_OPTIONS_H +typedef enum { + F_FOREGROUND = 0x1, + F_NOCCS = 0x2, + F_ERR = 0x4, + F_HELP = 0x8, + F_USE_UUID = 0x10, + F_VERSION = 0x20, + F_CCSERR = 0x40, + F_CCSFAIL = 0x80 +} arg_flags_t; + + typedef struct { char *addr; char *domain; @@ -30,15 +42,19 @@ int family; int timeout; int retr_time; -#define F_FOREGROUND 0x1 -#define F_DEBUG 0x2 -#define F_ERR 0x4 -#define F_HELP 0x8 -#define F_USE_UUID 0x10 -#define F_VERSION 0x20 - int flags; + arg_flags_t flags; + int debug; } fence_xvm_args_t; +/* Private structure for commandline / stdin fencing args */ +struct arg_info { + char opt; + char *opt_desc; + char *stdin_opt; + char *desc; + void (*assign)(fence_xvm_args_t *, struct arg_info *, char *); +}; + /* Get options */ void args_init(fence_xvm_args_t *args); @@ -47,6 +63,7 @@ void args_get_getopt(int argc, char **argv, char *optstr, fence_xvm_args_t *args); void args_get_stdin(char *optstr, fence_xvm_args_t *args); +void args_get_ccs(char *optstr, fence_xvm_args_t *args); void args_usage(char *progname, char *optstr, int print_stdin); void args_print(fence_xvm_args_t *args); --- cluster/fence/agents/xvm/xvm.h 2006/10/05 16:11:36 1.1 +++ cluster/fence/agents/xvm/xvm.h 2006/11/13 16:14:06 1.1.2.1 @@ -23,7 +23,7 @@ #include #include -#define XVM_VERSION "0.9.0" +#define XVM_VERSION "0.9.3" #define MAX_DOMAINNAME_LENGTH 64 /* XXX MAXHOSTNAMELEN */ #define MAX_ADDR_LEN sizeof(struct sockaddr_in6) @@ -72,4 +72,15 @@ uint8_t hash[MAX_HASH_LENGTH]; /* Binary hash */ } fence_req_t; + +inline void dset(int); +inline int dget(void); + +#define dprintf(level, fmt, args...) \ +do { \ + if (dget()>=level) \ + printf(fmt, ##args); \ +} while(0) + + #endif --- cluster/fence/agents/xvm/Makefile 2006/11/03 15:58:56 1.3 +++ cluster/fence/agents/xvm/Makefile 2006/11/13 16:14:06 1.3.2.1 @@ -16,9 +16,9 @@ TARGETS=fence_xvm fence_xvmd fence_xvm_SOURCE = fence_xvm.c mcast.c ip_lookup.c simple_auth.c tcp.c \ - options.c + options.c debug.c fence_xvmd_SOURCE= fence_xvmd.c mcast.c simple_auth.c tcp.c virt.c \ - options.c vm_states.c + options.c options-ccs.c vm_states.c debug.c INCLUDE=-I${top_srcdir}/include -I${top_srcdir}/config \ --- cluster/fence/agents/xvm/TODO 2006/10/05 16:11:36 1.1 +++ cluster/fence/agents/xvm/TODO 2006/11/13 16:14:06 1.1.2.1 @@ -4,9 +4,6 @@ Medium Priority: -* Enable retrieval of options from ccs rather than only on the -command line for fence_xvmd. - * Need to add ability for fence_xvmd to forcefully fence the host dom0 if it's not responding. Medium because it should not be the default behavior since fencing a host can affect multiple domains @@ -27,12 +24,6 @@ * Add SSL connection support. (Challenge/response on a trusted network should be okay.) -* Destroy/Create doesn't work very well with pygrub-loaded or -xenguest-install images. (Maybe just wait for solid virDomainReboot -operation based on Xen 3.0.4 API). For now, rely on external -management to restart the domain (a person, VM manager, or failover -manager like rgmanager) - * Make sure addresses contained in the multicast packet are always in network-byte order. Low because it will be unlikely that the host-byte ordering of a domU and its dom0 will be different. --- cluster/fence/agents/xvm/options.c 2006/11/03 15:58:56 1.2 +++ cluster/fence/agents/xvm/options.c 2006/11/13 16:14:06 1.2.2.1 @@ -43,15 +43,6 @@ #include "options.h" -/* Private structure for commandline / stdin fencing args */ -struct arg_info { - char opt; - char *opt_desc; - char *stdin_opt; - char *desc; - void (*assign)(fence_xvm_args_t *, struct arg_info *, char *); -}; - /* Assignment functions */ @@ -61,11 +52,14 @@ if (!value) { /* GNU getopt sets optarg to NULL for options w/o a param We rely on this here... */ - args->flags |= F_DEBUG; + args->debug++; return; } - args->flags |= ( !!atoi(value) ? F_DEBUG : 0); + args->debug = atoi(value); + if (args->debug < 0) { + args->debug = 1; + } } @@ -252,6 +246,13 @@ } +static inline void +assign_noccs(fence_xvm_args_t *args, struct arg_info *arg, char *value) +{ + args->flags |= F_NOCCS; +} + + /** ALL valid command line and stdin arguments for this fencing agent */ static struct arg_info _arg_info[] = { { '\xff', NULL, "agent", @@ -322,17 +323,20 @@ "Help (alternate)", assign_help }, + { 'X', "-X", NULL, + "Do not connect to CCS for configuration", + assign_noccs }, + { 'V', "-V", NULL, "Display version and exit", assign_version }, - /* Terminator */ { 0, NULL, NULL, NULL, NULL } }; -static struct arg_info * +struct arg_info * find_arg_by_char(char arg) { int x = 0; @@ -346,7 +350,7 @@ } -static struct arg_info * +struct arg_info * find_arg_by_string(char *arg) { int x = 0; @@ -383,6 +387,7 @@ args->timeout = 30; args->retr_time = 20; args->flags = 0; + args->debug = 0; } @@ -410,6 +415,7 @@ _pr_int(args->timeout); _pr_int(args->retr_time); _pr_int(args->flags); + _pr_int(args->debug); printf("-- end args --\n"); } @@ -475,7 +481,7 @@ { char *p; int x; - + /* Remove leading whitespace. */ p = line; for (x = 0; x <= linelen; x++) { --- cluster/fence/agents/xvm/vm_states.c 2006/10/05 16:11:36 1.1 +++ cluster/fence/agents/xvm/vm_states.c 2006/11/13 16:14:06 1.1.2.1 @@ -238,9 +238,10 @@ err = saCkptInitialize(&h->ck_handle, NULL, &ver); - if (err != SA_AIS_OK) + if (err != SA_AIS_OK) { free(h); - else + return NULL; + } else h->ck_ready = READY_MAGIC; if (ckpt_open(h, ckpt_name, maxlen, maxsec, maxseclen, --- cluster/fence/agents/xvm/tcp.c 2006/10/05 16:11:36 1.1 +++ cluster/fence/agents/xvm/tcp.c 2006/11/13 16:14:06 1.1.2.1 @@ -65,6 +65,7 @@ struct sockaddr_in6 _sin6; int fd, ret; + dprintf(4, "%s: Setting up ipv6 listen socket\n", __FUNCTION__); fd = socket(PF_INET6, SOCK_STREAM, 0); if (fd < 0) return -1; @@ -95,6 +96,7 @@ return -1; } + dprintf(4, "%s: Success; fd = %d\n", __FUNCTION__, fd); return fd; } @@ -113,6 +115,7 @@ struct sockaddr_in _sin; int fd, ret; + dprintf(4, "%s: Setting up ipv4 listen socket\n", __FUNCTION__); fd = socket(PF_INET, SOCK_STREAM, 0); if (fd < 0) return -1; @@ -141,6 +144,7 @@ return -1; } + dprintf(4, "%s: Success; fd = %d\n", __FUNCTION__, fd); return fd; } @@ -162,6 +166,7 @@ struct sockaddr_in6 _sin6; int fd, ret; + dprintf(4, "%s: Connecting to client\n", __FUNCTION__); fd = socket(PF_INET6, SOCK_STREAM, 0); if (fd < 0) return -1; @@ -177,6 +182,7 @@ close(fd); return -1; } + dprintf(4, "%s: Success; fd = %d\n", __FUNCTION__, fd); return fd; } @@ -197,6 +203,7 @@ struct sockaddr_in _sin; int fd, ret; + dprintf(4, "%s: Connecting to client\n", __FUNCTION__); fd = socket(PF_INET, SOCK_STREAM, 0); if (fd < 0) return -1; @@ -211,6 +218,7 @@ return -1; } + dprintf(4, "%s: Success; fd = %d\n", __FUNCTION__, fd); return fd; } --- cluster/fence/agents/xvm/fence_xvmd.c 2006/11/03 15:58:56 1.4 +++ cluster/fence/agents/xvm/fence_xvmd.c 2006/11/13 16:14:06 1.4.2.1 @@ -200,6 +200,9 @@ return; end += strlen(ENDOSTAG); + dprintf(3, "Clearing %d bytes starting @ %p\n", (int)(end-start), + start); + memset(start, ' ', end-start); } @@ -213,12 +216,17 @@ char response = 1; char *domain_desc; - if (!(vdp = get_domain(req, vp))) + if (!(vdp = get_domain(req, vp))) { + dprintf(2, "Could not find domain: %s\n", req->domain); goto out; + } fd = connect_tcp(req, auth, key, key_len); - if (fd < 0) + if (fd < 0) { + dprintf(2, "Could call back for fence request: %s\n", + strerror(errno)); goto out; + } switch(req->request) { case FENCE_NULL: @@ -252,10 +260,21 @@ (char *)req->domain); domain_desc = virDomainGetXMLDesc(vdp, 0); - if (domain_desc) + if (domain_desc) { + dprintf(3, "[[ XML Domain Info ]]\n"); + dprintf(3, "%s\n[[ XML END ]]\n", domain_desc); cleanup_xmldesc(domain_desc); + dprintf(3, "[[ XML Domain Info (modified) ]]\n"); + dprintf(3, "%s\n[[ XML END ]]\n", domain_desc); + } else { + printf("Failed getting domain description from " + "libvirt\n"); + } + + dprintf(2, "Calling virDomainDestroy\n"); ret = virDomainDestroy(vdp); if (ret < 0) { + printf("virDomainDestroy() failed: %d\n", ret); if (domain_desc) free(domain_desc); break; @@ -271,8 +290,10 @@ be necessary */ vdp = get_domain(req, vp); if (!vdp) { + dprintf(2, "Domain no longer exists\n"); response = 0; /* Success! */ } else { + printf("Domain still exists; fencing failed\n"); virDomainFree(vdp); ret = 1; /* Failed to kill it */ } @@ -280,6 +301,7 @@ /* Recreate the domain if possible */ if (ret == 0 && domain_desc) { /* Success */ + dprintf(2, "Calling virDomainCreateLinux()...\n"); virDomainCreateLinux(vp, domain_desc, 0); } @@ -288,6 +310,7 @@ break; } + dprintf(3, "Sending response to caller...\n"); if (write(fd, &response, 1) < 0) { perror("write"); } @@ -622,15 +645,20 @@ fence_xvm_args_t args; int mc_sock; char key[4096]; - int key_len; - char *my_options = "dfi:a:p:C:c:k:u?hV"; + int key_len = 0; + char *my_options = "dfi:a:p:C:c:k:u?hVX"; void *h; args_init(&args); args_get_getopt(argc, argv, my_options, &args); + if (!(args.flags & F_NOCCS)) { + args_get_ccs(my_options, &args); + } args_finalize(&args); - if (args.flags & F_DEBUG) + if (args.debug > 0) { + dset(args.debug); args_print(&args); + } if (args.flags & F_ERR) { args_usage(argv[0], my_options, 0); @@ -650,10 +678,12 @@ exit(0); } - key_len = read_key_file(args.key_file, key, sizeof(key)); - if (key_len < 0) { - printf("Could not read key file\n"); - return 1; + if (args.auth != AUTH_NONE || args.hash != HASH_NONE) { + key_len = read_key_file(args.key_file, key, sizeof(key)); + if (key_len < 0) { + printf("Could not read key file\n"); + return 1; + } } /* Fork in to background */ --- cluster/fence/agents/xvm/mcast.c 2006/10/05 16:11:36 1.1 +++ cluster/fence/agents/xvm/mcast.c 2006/11/13 16:14:06 1.1.2.1 @@ -62,6 +62,7 @@ /******************************** * SET UP MULTICAST RECV SOCKET * ********************************/ + dprintf(4, "Setting up ipv4 multicast receive (%s:%d)\n", addr, port); sock = socket(PF_INET, SOCK_DGRAM, 0); if (sock < 0) { printf("socket: %s\n", strerror(errno)); @@ -89,6 +90,7 @@ */ /* mreq.imr_multiaddr.s_addr is set above */ mreq.imr_interface.s_addr = htonl(INADDR_ANY); + dprintf(4, "Joining multicast group\n"); if (setsockopt(sock, IPPROTO_IP, IP_ADD_MEMBERSHIP, &mreq, sizeof(mreq)) == -1) { printf("Failed to bind multicast receive socket to " @@ -98,6 +100,7 @@ return -1; } + dprintf(4, "%s: success, fd = %d\n", __FUNCTION__, sock); return sock; } @@ -144,6 +147,7 @@ /************************* * SET UP MULTICAST SEND * *************************/ + dprintf(4, "Setting up ipv4 multicast send (%s:%d)\n", addr, port); sock = socket(PF_INET, SOCK_DGRAM, 0); if (sock < 0) { perror("socket"); @@ -153,6 +157,7 @@ /* * Join Multicast group. */ + dprintf(4, "Joining IP Multicast group (pass 1)\n"); if (setsockopt(sock, IPPROTO_IP, IP_ADD_MEMBERSHIP, &mreq, sizeof(mreq)) == -1) { printf("Failed to add multicast membership to transmit " @@ -164,6 +169,7 @@ /* * Join Multicast group. */ + dprintf(4, "Joining IP Multicast group (pass 2)\n"); if (setsockopt(sock, IPPROTO_IP, IP_MULTICAST_IF, &src.sin_addr, sizeof(src.sin_addr)) == -1) { printf("Failed to bind multicast transmit socket to " @@ -175,6 +181,7 @@ /* * set time to live to 2 hops. */ + dprintf(4, "Setting TTL to 2 for fd%d\n", sock); val = 2; if (setsockopt(sock, SOL_IP, IP_MULTICAST_TTL, &val, sizeof(val))) @@ -182,6 +189,7 @@ memcpy((struct sockaddr_in *)tgt, &mcast, sizeof(struct sockaddr_in)); + dprintf(4, "%s: success, fd = %d\n", __FUNCTION__, sock); return sock; } @@ -214,6 +222,7 @@ /******************************** * SET UP MULTICAST RECV SOCKET * ********************************/ + dprintf(4, "Setting up ipv6 multicast receive (%s:%d)\n", addr, port); sock = socket(PF_INET6, SOCK_DGRAM, IPPROTO_UDP); if (sock < 0) { printf("socket: %s\n", strerror(errno)); @@ -237,6 +246,7 @@ return -1; } + dprintf(4, "Disabling IP Multicast loopback\n"); val = 1; if (setsockopt(sock, IPPROTO_IPV6, IPV6_MULTICAST_LOOP, &val, sizeof(val)) != 0) { @@ -248,6 +258,7 @@ /* * Join multicast group */ + dprintf(4, "Joining IP Multicast group\n"); if (setsockopt(sock, IPPROTO_IPV6, IPV6_ADD_MEMBERSHIP, &mreq, sizeof(mreq)) == -1) { printf("Failed to add multicast to socket %s: %s\n", @@ -256,6 +267,7 @@ return -1; } + dprintf(4, "%s: success, fd = %d\n", __FUNCTION__, sock); return sock; } @@ -304,12 +316,14 @@ /************************* * SET UP MULTICAST SEND * *************************/ + dprintf(4, "Setting up ipv6 multicast send (%s:%d)\n", addr, port); sock = socket(PF_INET6, SOCK_DGRAM, 0); if (sock < 0) { perror("socket"); return -1; } + dprintf(4, "Disabling IP Multicast loopback\n"); val = 1; if (setsockopt(sock, IPPROTO_IPV6, IPV6_MULTICAST_LOOP, &val, sizeof(val)) != 0) { @@ -321,6 +335,7 @@ /* * Join Multicast group. */ + dprintf(4, "Joining IP Multicast group\n"); if (setsockopt(sock, IPPROTO_IPV6, IPV6_ADD_MEMBERSHIP, &mreq, sizeof(mreq)) == -1) { printf("Failed to add multicast membership to transmit " @@ -352,5 +367,6 @@ memcpy((struct sockaddr_in *)tgt, &mcast, sizeof(struct sockaddr_in6)); + dprintf(4, "%s: success, fd = %d\n", __FUNCTION__, sock); return sock; } --- cluster/fence/agents/xvm/ip_lookup.c 2006/11/01 18:29:19 1.2 +++ cluster/fence/agents/xvm/ip_lookup.c 2006/11/13 16:14:06 1.2.2.1 @@ -84,6 +84,8 @@ if (!strncmp(ipaddr, "feb0", 4)) return -1; } + + dprintf(4, "Adding IP %s to list (family %d)\n", ipaddr, family); ipa = malloc(sizeof(*ipa)); memset(ipa, 0, sizeof(*ipa)); @@ -108,19 +110,25 @@ char outbuf[256]; int x, fd, len; + dprintf(5, "Connecting to Netlink...\n"); fd = socket(PF_NETLINK, SOCK_DGRAM, NETLINK_ROUTE); if (fd < 0) { perror("socket"); exit(1); } - + + dprintf(5, "Sending address dump request\n"); send_addr_dump(fd, family); memset(buf, 0, sizeof(buf)); + + dprintf(5, "Waiting for response\n"); x = recvfrom(fd, buf, sizeof(buf), 0, NULL, 0); if (x < 0) { perror("recvfrom"); return -1; } + + dprintf(5, "Received %d bytes\n", x); nh = (struct nlmsghdr *)buf; while (NLMSG_OK(nh, x)) { @@ -164,8 +172,10 @@ len -= sizeof(*ifa); do { /* Make sure we've got a valid rtaddr field */ - if (!RTA_OK(rta, len)) + if (!RTA_OK(rta, len)) { + dprintf(5, "!RTA_OK(rta, len)\n"); break; + } if (rta->rta_type == IFA_ADDRESS) { inet_ntop(family, RTA_DATA(rta), outbuf, @@ -174,7 +184,8 @@ } if (rta->rta_type == IFA_LABEL) { - printf("label: %s\n", (char *)RTA_DATA(rta)); + dprintf(5, "Skipping label: %s\n", + (char *)RTA_DATA(rta)); } nrta = RTA_NEXT(rta, len); @@ -188,6 +199,7 @@ nh = NLMSG_NEXT(nh, x); } + dprintf(5, "Closing Netlink connection\n"); close(fd); return 0; } @@ -197,13 +209,16 @@ ip_search(ip_list_t *ipl, char *ip_name) { ip_addr_t *ipa; - + + dprintf(5, "Looking for IP address %s in IP list %p...", ip_name, ipl); ipa = ipl->tqh_first; for (ipa = ipl->tqh_first; ipa; ipa = ipa->ipa_entries.tqe_next) { if (!strcmp(ip_name, ipa->ipa_address)) { + dprintf(4,"Found\n"); return 0; } } + dprintf(5, "Not found\n"); return 1; } @@ -212,7 +227,8 @@ ip_free_list(ip_list_t *ipl) { ip_addr_t *ipa; - + + dprintf(5, "Tearing down IP list @ %p\n", ipl); while ((ipa = ipl->tqh_first)) { TAILQ_REMOVE(ipl, ipa, ipa_entries); free(ipa->ipa_address); @@ -225,6 +241,7 @@ int ip_build_list(ip_list_t *ipl) { + dprintf(5, "Build IP address list\n"); TAILQ_INIT(ipl); if (add_ip_addresses(PF_INET6, ipl) < 0) { ip_free_list(ipl); @@ -258,6 +275,7 @@ ip_list_t ipl; int ret = -1; + dprintf(5, "Looking for IP matching %s\n", nodename); /* Build list of IP addresses configured locally */ if (ip_build_list(&ipl) < 0) return -1; @@ -265,6 +283,7 @@ /* Get list of addresses for the host-name/ip */ if (getaddrinfo(nodename, NULL, NULL, &ai) != 0) return -1; + /* Traverse list of addresses for given host-name/ip */ for (n = ai; n; n = n->ai_next) { --- cluster/fence/agents/xvm/simple_auth.c 2006/10/05 16:11:36 1.1 +++ cluster/fence/agents/xvm/simple_auth.c 2006/11/13 16:14:06 1.1.2.1 @@ -24,6 +24,7 @@ #include #include #include +#include /* Local includes */ #include "xvm.h" @@ -63,6 +64,7 @@ return; } + dprintf(4, "Opening /dev/urandom\n"); devrand = open("/dev/urandom", O_RDONLY); if (devrand >= 0) { if (read(devrand, req->random, sizeof(req->random)) < 0) { @@ -107,6 +109,7 @@ ht = HASH_AlgSHA512; break; default: + dprintf(3, "%s: no-op (HASH_NONE)\n", __FUNCTION__); return 0; } @@ -145,6 +148,7 @@ memset(req->hash, 0, sizeof(req->hash)); switch(req->hashtype) { case HASH_NONE: + dprintf(3, "%s: no-op (HASH_NONE)\n", __FUNCTION__); return 0; case HASH_SHA1: case HASH_SHA256: @@ -301,6 +305,7 @@ ht = HASH_AlgSHA512; break; default: + dprintf(3, "%s: no-op (AUTH_NONE)\n", __FUNCTION__); return 0; } @@ -330,6 +335,7 @@ { switch(auth) { case AUTH_NONE: + dprintf(3, "%s: no-op (AUTH_NONE)\n", __FUNCTION__); return 1; case AUTH_SHA1: case AUTH_SHA256: @@ -348,6 +354,7 @@ { switch(auth) { case AUTH_NONE: + dprintf(3, "%s: no-op (AUTH_NONE)\n", __FUNCTION__); return 1; case AUTH_SHA1: case AUTH_SHA256: @@ -367,6 +374,8 @@ int nread, remain = max_len; char *p; + dprintf(3, "Reading in key file %s into %p (%d len)", + file, key, max_len); fd = open(file, O_RDONLY); if (fd < 0) { return -1; @@ -379,17 +388,22 @@ while (remain) { nread = read(fd, p, remain); if (nread < 0) { + dprintf(2, "Error from read: %s\n", strerror(errno)); close(fd); return -1; } - if (nread == 0) + if (nread == 0) { + dprintf(3, "Stopped reading @ %d bytes", + max_len-remain); break; + } p += nread; remain -= nread; } + dprintf(3, "Actual key length = %d bytes", max_len-remain); close(fd); return 0; --- cluster/cman/init.d/cman 2006/10/30 13:20:27 1.26 +++ cluster/cman/init.d/cman 2006/11/13 16:14:06 1.26.2.1 @@ -123,6 +123,43 @@ return 0 } +start_fence_xvmd() +{ + status fence_xvmd &> /dev/null + if [ $? -ne 0 ]; then + errmsg=$( /sbin/fence_xvmd $FENCE_XVMD_OPTS 2>&1 ) || return 1 + fi + return 0 +} + + +fence_xvmd_enabled() +{ + # + # Check for the 'xm' binary. If it's not here, we are not + # running on a machine capable of running xvmd. + # + which xm &> /dev/null || return 1 + + # + # Check for presence of Domain-0; if it's not there, we can't + # run xvmd. + # + xm list --long | grep -q "Domain-0" || return 1 + + # + # Check for presence of /cluster/fence_xvmd in cluster.conf + # (If -X is specified, it doesn't matter if it's in cluster.conf; + # we'll start it anyway since ccsd is not required) + # + if [ "$FENCE_XVMD_OPTS" = "${FENCE_XVMD_OPTS/-X/}" ]; then + xmllint --shell /etc/cluster/cluster.conf 2> /dev/null \ + < <(echo ls cluster) | grep -q fence_xvmd || return 1 + fi + + return 0 +} + start() { echo "Starting cluster: " @@ -182,6 +219,18 @@ return 1 fi + if fence_xvmd_enabled; then + echo -n " Starting virtual machine fencing host... " + start_fence_xvmd + if [ $? -eq 0 ] + then + echo "done" + else + echo "failed" + return 1 + fi + fi + return 0 } @@ -258,9 +307,32 @@ return 0 # all ok } +stop_fence_xvmd() +{ + if /sbin/pidof fence_xvmd &> /dev/null + then + pkill -TERM fence_xvmd + sleep 1 # A bit of time for fenced to exit + fi + + [ -z "`pidof fence_xvmd`" ] + return $? +} + stop() { echo "Stopping cluster: " + if fence_xvmd_enabled; then + echo -n " Stopping virtual machine fencing host... " + stop_fence_xvmd + if [ $? -eq 0 ] + then + echo "done" + else + echo "failed" + return 1 + fi + fi echo -n " Stopping fencing... " stop_fence if [ $? -eq 0 ] @@ -314,6 +386,10 @@ errmsg=$( status fenced 2>&1) || return 1 errmsg=$( status dlm_controld 2>&1) || return 1 errmsg=$( status gfs_controld 2>&1) || return 1 + + fence_xvmd_enabled || return 0 + errmsg=$( status fence_xvmd 2>&1) || return 1 + return 0 } --- cluster/fence/man/fence_xvmd.8 2006/10/05 16:11:36 1.1 +++ cluster/fence/man/fence_xvmd.8 2006/11/13 16:14:08 1.1.2.1 @@ -33,7 +33,8 @@ Foreground mode (do not fork) .TP \fB-d\fP -Enable debugging output +Enable debugging output. The more times you specify this parameter, +the more debugging output you will receive. .TP \fB-i\fP \fIfamily\fP IP family to use (auto, ipv4, or ipv6; default = auto) @@ -61,6 +62,8 @@ .TP \fB-k\fP \fIkey_file\fP Use the specified key file for packet hashing / SHA authentication. +When both the hash type and the authentication type are set to "none", +this parameter is ignored. .TP \fB-u\fP Fence by UUID instead of Xen Domain name. @@ -71,8 +74,44 @@ \fB-h\fP Print out a help message describing available options, then exit. .TP +\fB-X\fP +Do not connect to CCS for configuration; only use command line +parameters. CCS configuration parameters override command line +parameters (because they are cluster-wide), so if you need to +override a configuration option contained in CCS, you must specify +this parameter. \fB-V\fP Print out a version message, then exit. +.SH CCS PARAMETERS +CCS options are simply attributes of the tag, a +child of the tag in /etc/cluster/cluster.conf. +.TP +\fIdebug="1"\fR +Same as the -d option. Specify numbers >1 for more debugging information. +.TP +\fIfamily="param"\fR +Same as the -i option. +.TP +\fImulticast_address="param"\fR +Same as the -a option. +.TP +\fIport="param"\fR +Same as the -p option. +.TP +\fIauth="param"\fR +Same as the -C option. +.TP +\fIhash="param"\fR +Same as the -c option. +.TP +\fIkey_file="param"\fR +Same as the -k option. +.TP +\fIuse_uuid="1"\fR +Same as the -u option. +.TP + + .SH SEE ALSO fence(8), fence_node(8), fence_xvm(8) --- cluster/fence/man/fence_xvm.8 2006/10/05 16:11:36 1.1 +++ cluster/fence/man/fence_xvm.8 2006/11/13 16:14:08 1.1.2.1 @@ -30,7 +30,8 @@ .SH OPTIONS .TP \fB-d\fP -Enable debugging output +Enable debugging output. The more times you specify this parameter, +the more debugging output you will receive. .TP \fB-i\fP \fIfamily\fP IP family to use (auto, ipv4, or ipv6; default = auto) @@ -62,6 +63,8 @@ .TP \fB-k\fP \fIkey_file\fP Use the specified key file for packet hashing / SHA authentication. +When both the hash type and the authentication type are set to "none", +this parameter is ignored. .TP \fB-H\fP \fIdomain\fP This specifies unique domain name of the Xen guest which needs to be fenced. @@ -94,7 +97,7 @@ .SH STDIN PARAMETERS .TP \fIdebug = 1\fR -Same as the -d option. +Same as the -d option. Specify numbers >1 for more debugging information. .TP \fIfamily = < param >\fR Same as the -i option.