* Re: [1.8.0] split largest remaining scripts, gitk and gitweb
From: Jakub Narebski @ 2011-02-01 23:20 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Nicolas Pitre, git
In-Reply-To: <7vsjw7xuy3.fsf@alter.siamese.dyndns.org>
Dnia wtorek 1. lutego 2011 23:15, Junio C Hamano napisał:
> Jakub Narebski <jnareb@gmail.com> writes:
>
> > Two largest files in git repository are gitk and gitweb, see the
> > ...
> > What do you think?
>
> What does this have to do anything with 1.8.0? Isn't this all internal
> implementation that can be brought in without affecting end users?
In the case of gitk we have "prior art" i.e. git-gui, which got split.
In the case of gitweb I am not sure if there having multiple files to
install wouldn't be inconvenient for users (though with "install-gitweb"
target...)
Anyway, I have posted this in subthread of
[1.8.0] reorganize the mess that the source tree has become
because it is also code reorganization.
--
Jakub Narebski
Poland
^ permalink raw reply
* [PATCH] ipv4: Remove fib_hash.
From: David Miller @ 2011-02-01 23:19 UTC (permalink / raw)
To: netdev
The time has finally come to remove the hash based routing table
implementation in ipv4.
FIB Trie is mature, well tested, and I've done an audit of it's code
to confirm that it implements insert, delete, and lookup with the same
identical semantics as fib_hash did.
If there are any semantic differences found in fib_trie, we should
simply fix them.
I've placed the trie statistic config option under advanced router
configuration.
Signed-off-by: David S. Miller <davem@davemloft.net>
---
net/ipv4/Kconfig | 38 +--
net/ipv4/Makefile | 4 +-
net/ipv4/fib_hash.c | 1061 ---------------------------------------------------
3 files changed, 2 insertions(+), 1101 deletions(-)
delete mode 100644 net/ipv4/fib_hash.c
diff --git a/net/ipv4/Kconfig b/net/ipv4/Kconfig
index 8949a05..cbb505b 100644
--- a/net/ipv4/Kconfig
+++ b/net/ipv4/Kconfig
@@ -55,45 +55,9 @@ config IP_ADVANCED_ROUTER
If unsure, say N here.
-choice
- prompt "Choose IP: FIB lookup algorithm (choose FIB_HASH if unsure)"
- depends on IP_ADVANCED_ROUTER
- default ASK_IP_FIB_HASH
-
-config ASK_IP_FIB_HASH
- bool "FIB_HASH"
- ---help---
- Current FIB is very proven and good enough for most users.
-
-config IP_FIB_TRIE
- bool "FIB_TRIE"
- ---help---
- Use new experimental LC-trie as FIB lookup algorithm.
- This improves lookup performance if you have a large
- number of routes.
-
- LC-trie is a longest matching prefix lookup algorithm which
- performs better than FIB_HASH for large routing tables.
- But, it consumes more memory and is more complex.
-
- LC-trie is described in:
-
- IP-address lookup using LC-tries. Stefan Nilsson and Gunnar Karlsson
- IEEE Journal on Selected Areas in Communications, 17(6):1083-1092,
- June 1999
-
- An experimental study of compression methods for dynamic tries
- Stefan Nilsson and Matti Tikkanen. Algorithmica, 33(1):19-33, 2002.
- <http://www.csc.kth.se/~snilsson/software/dyntrie2/>
-
-endchoice
-
-config IP_FIB_HASH
- def_bool ASK_IP_FIB_HASH || !IP_ADVANCED_ROUTER
-
config IP_FIB_TRIE_STATS
bool "FIB TRIE statistics"
- depends on IP_FIB_TRIE
+ depends on IP_ADVANCED_ROUTER
---help---
Keep track of statistics on structure of FIB TRIE table.
Useful for testing and measuring TRIE performance.
diff --git a/net/ipv4/Makefile b/net/ipv4/Makefile
index 4978d22..0dc772d 100644
--- a/net/ipv4/Makefile
+++ b/net/ipv4/Makefile
@@ -10,12 +10,10 @@ obj-y := route.o inetpeer.o protocol.o \
tcp_minisocks.o tcp_cong.o \
datagram.o raw.o udp.o udplite.o \
arp.o icmp.o devinet.o af_inet.o igmp.o \
- fib_frontend.o fib_semantics.o \
+ fib_frontend.o fib_semantics.o fib_trie.o \
inet_fragment.o
obj-$(CONFIG_SYSCTL) += sysctl_net_ipv4.o
-obj-$(CONFIG_IP_FIB_HASH) += fib_hash.o
-obj-$(CONFIG_IP_FIB_TRIE) += fib_trie.o
obj-$(CONFIG_PROC_FS) += proc.o
obj-$(CONFIG_IP_MULTIPLE_TABLES) += fib_rules.o
obj-$(CONFIG_IP_MROUTE) += ipmr.o
diff --git a/net/ipv4/fib_hash.c b/net/ipv4/fib_hash.c
deleted file mode 100644
index fadb602..0000000
--- a/net/ipv4/fib_hash.c
+++ /dev/null
@@ -1,1061 +0,0 @@
-/*
- * INET An implementation of the TCP/IP protocol suite for the LINUX
- * operating system. INET is implemented using the BSD Socket
- * interface as the means of communication with the user level.
- *
- * IPv4 FIB: lookup engine and maintenance routines.
- *
- * Authors: Alexey Kuznetsov, <kuznet@ms2.inr.ac.ru>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- */
-
-#include <asm/uaccess.h>
-#include <asm/system.h>
-#include <linux/bitops.h>
-#include <linux/types.h>
-#include <linux/kernel.h>
-#include <linux/mm.h>
-#include <linux/string.h>
-#include <linux/socket.h>
-#include <linux/sockios.h>
-#include <linux/errno.h>
-#include <linux/in.h>
-#include <linux/inet.h>
-#include <linux/inetdevice.h>
-#include <linux/netdevice.h>
-#include <linux/if_arp.h>
-#include <linux/proc_fs.h>
-#include <linux/skbuff.h>
-#include <linux/netlink.h>
-#include <linux/init.h>
-#include <linux/slab.h>
-
-#include <net/net_namespace.h>
-#include <net/ip.h>
-#include <net/protocol.h>
-#include <net/route.h>
-#include <net/tcp.h>
-#include <net/sock.h>
-#include <net/ip_fib.h>
-
-#include "fib_lookup.h"
-
-static struct kmem_cache *fn_hash_kmem __read_mostly;
-static struct kmem_cache *fn_alias_kmem __read_mostly;
-
-struct fib_node {
- struct hlist_node fn_hash;
- struct list_head fn_alias;
- __be32 fn_key;
- struct fib_alias fn_embedded_alias;
-};
-
-#define EMBEDDED_HASH_SIZE (L1_CACHE_BYTES / sizeof(struct hlist_head))
-
-struct fn_zone {
- struct fn_zone __rcu *fz_next; /* Next not empty zone */
- struct hlist_head __rcu *fz_hash; /* Hash table pointer */
- seqlock_t fz_lock;
- u32 fz_hashmask; /* (fz_divisor - 1) */
-
- u8 fz_order; /* Zone order (0..32) */
- u8 fz_revorder; /* 32 - fz_order */
- __be32 fz_mask; /* inet_make_mask(order) */
-#define FZ_MASK(fz) ((fz)->fz_mask)
-
- struct hlist_head fz_embedded_hash[EMBEDDED_HASH_SIZE];
-
- int fz_nent; /* Number of entries */
- int fz_divisor; /* Hash size (mask+1) */
-};
-
-struct fn_hash {
- struct fn_zone *fn_zones[33];
- struct fn_zone __rcu *fn_zone_list;
-};
-
-static inline u32 fn_hash(__be32 key, struct fn_zone *fz)
-{
- u32 h = ntohl(key) >> fz->fz_revorder;
- h ^= (h>>20);
- h ^= (h>>10);
- h ^= (h>>5);
- h &= fz->fz_hashmask;
- return h;
-}
-
-static inline __be32 fz_key(__be32 dst, struct fn_zone *fz)
-{
- return dst & FZ_MASK(fz);
-}
-
-static unsigned int fib_hash_genid;
-
-#define FZ_MAX_DIVISOR ((PAGE_SIZE<<MAX_ORDER) / sizeof(struct hlist_head))
-
-static struct hlist_head *fz_hash_alloc(int divisor)
-{
- unsigned long size = divisor * sizeof(struct hlist_head);
-
- if (size <= PAGE_SIZE)
- return kzalloc(size, GFP_KERNEL);
-
- return (struct hlist_head *)
- __get_free_pages(GFP_KERNEL | __GFP_ZERO, get_order(size));
-}
-
-/* The fib hash lock must be held when this is called. */
-static inline void fn_rebuild_zone(struct fn_zone *fz,
- struct hlist_head *old_ht,
- int old_divisor)
-{
- int i;
-
- for (i = 0; i < old_divisor; i++) {
- struct hlist_node *node, *n;
- struct fib_node *f;
-
- hlist_for_each_entry_safe(f, node, n, &old_ht[i], fn_hash) {
- struct hlist_head *new_head;
-
- hlist_del_rcu(&f->fn_hash);
-
- new_head = rcu_dereference_protected(fz->fz_hash, 1) +
- fn_hash(f->fn_key, fz);
- hlist_add_head_rcu(&f->fn_hash, new_head);
- }
- }
-}
-
-static void fz_hash_free(struct hlist_head *hash, int divisor)
-{
- unsigned long size = divisor * sizeof(struct hlist_head);
-
- if (size <= PAGE_SIZE)
- kfree(hash);
- else
- free_pages((unsigned long)hash, get_order(size));
-}
-
-static void fn_rehash_zone(struct fn_zone *fz)
-{
- struct hlist_head *ht, *old_ht;
- int old_divisor, new_divisor;
- u32 new_hashmask;
-
- new_divisor = old_divisor = fz->fz_divisor;
-
- switch (old_divisor) {
- case EMBEDDED_HASH_SIZE:
- new_divisor *= EMBEDDED_HASH_SIZE;
- break;
- case EMBEDDED_HASH_SIZE*EMBEDDED_HASH_SIZE:
- new_divisor *= (EMBEDDED_HASH_SIZE/2);
- break;
- default:
- if ((old_divisor << 1) > FZ_MAX_DIVISOR) {
- printk(KERN_CRIT "route.c: bad divisor %d!\n", old_divisor);
- return;
- }
- new_divisor = (old_divisor << 1);
- break;
- }
-
- new_hashmask = (new_divisor - 1);
-
-#if RT_CACHE_DEBUG >= 2
- printk(KERN_DEBUG "fn_rehash_zone: hash for zone %d grows from %d\n",
- fz->fz_order, old_divisor);
-#endif
-
- ht = fz_hash_alloc(new_divisor);
-
- if (ht) {
- struct fn_zone nfz;
-
- memcpy(&nfz, fz, sizeof(nfz));
-
- write_seqlock_bh(&fz->fz_lock);
- old_ht = rcu_dereference_protected(fz->fz_hash, 1);
- RCU_INIT_POINTER(nfz.fz_hash, ht);
- nfz.fz_hashmask = new_hashmask;
- nfz.fz_divisor = new_divisor;
- fn_rebuild_zone(&nfz, old_ht, old_divisor);
- fib_hash_genid++;
- rcu_assign_pointer(fz->fz_hash, ht);
- fz->fz_hashmask = new_hashmask;
- fz->fz_divisor = new_divisor;
- write_sequnlock_bh(&fz->fz_lock);
-
- if (old_ht != fz->fz_embedded_hash) {
- synchronize_rcu();
- fz_hash_free(old_ht, old_divisor);
- }
- }
-}
-
-static void fn_free_node_rcu(struct rcu_head *head)
-{
- struct fib_node *f = container_of(head, struct fib_node, fn_embedded_alias.rcu);
-
- kmem_cache_free(fn_hash_kmem, f);
-}
-
-static inline void fn_free_node(struct fib_node *f)
-{
- call_rcu(&f->fn_embedded_alias.rcu, fn_free_node_rcu);
-}
-
-static void fn_free_alias_rcu(struct rcu_head *head)
-{
- struct fib_alias *fa = container_of(head, struct fib_alias, rcu);
-
- kmem_cache_free(fn_alias_kmem, fa);
-}
-
-static inline void fn_free_alias(struct fib_alias *fa, struct fib_node *f)
-{
- fib_release_info(fa->fa_info);
- if (fa == &f->fn_embedded_alias)
- fa->fa_info = NULL;
- else
- call_rcu(&fa->rcu, fn_free_alias_rcu);
-}
-
-static struct fn_zone *
-fn_new_zone(struct fn_hash *table, int z)
-{
- int i;
- struct fn_zone *fz = kzalloc(sizeof(struct fn_zone), GFP_KERNEL);
- if (!fz)
- return NULL;
-
- seqlock_init(&fz->fz_lock);
- fz->fz_divisor = z ? EMBEDDED_HASH_SIZE : 1;
- fz->fz_hashmask = fz->fz_divisor - 1;
- RCU_INIT_POINTER(fz->fz_hash, fz->fz_embedded_hash);
- fz->fz_order = z;
- fz->fz_revorder = 32 - z;
- fz->fz_mask = inet_make_mask(z);
-
- /* Find the first not empty zone with more specific mask */
- for (i = z + 1; i <= 32; i++)
- if (table->fn_zones[i])
- break;
- if (i > 32) {
- /* No more specific masks, we are the first. */
- rcu_assign_pointer(fz->fz_next,
- rtnl_dereference(table->fn_zone_list));
- rcu_assign_pointer(table->fn_zone_list, fz);
- } else {
- rcu_assign_pointer(fz->fz_next,
- rtnl_dereference(table->fn_zones[i]->fz_next));
- rcu_assign_pointer(table->fn_zones[i]->fz_next, fz);
- }
- table->fn_zones[z] = fz;
- fib_hash_genid++;
- return fz;
-}
-
-int fib_table_lookup(struct fib_table *tb,
- const struct flowi *flp, struct fib_result *res,
- int fib_flags)
-{
- int err;
- struct fn_zone *fz;
- struct fn_hash *t = (struct fn_hash *)tb->tb_data;
-
- rcu_read_lock();
- for (fz = rcu_dereference(t->fn_zone_list);
- fz != NULL;
- fz = rcu_dereference(fz->fz_next)) {
- struct hlist_head *head;
- struct hlist_node *node;
- struct fib_node *f;
- __be32 k;
- unsigned int seq;
-
- do {
- seq = read_seqbegin(&fz->fz_lock);
- k = fz_key(flp->fl4_dst, fz);
-
- head = rcu_dereference(fz->fz_hash) + fn_hash(k, fz);
- hlist_for_each_entry_rcu(f, node, head, fn_hash) {
- if (f->fn_key != k)
- continue;
-
- err = fib_semantic_match(tb, &f->fn_alias,
- flp, res,
- fz->fz_order, fib_flags);
- if (err <= 0)
- goto out;
- }
- } while (read_seqretry(&fz->fz_lock, seq));
- }
- err = 1;
-out:
- rcu_read_unlock();
- return err;
-}
-
-/* Insert node F to FZ. */
-static inline void fib_insert_node(struct fn_zone *fz, struct fib_node *f)
-{
- struct hlist_head *head = rtnl_dereference(fz->fz_hash) + fn_hash(f->fn_key, fz);
-
- hlist_add_head_rcu(&f->fn_hash, head);
-}
-
-/* Return the node in FZ matching KEY. */
-static struct fib_node *fib_find_node(struct fn_zone *fz, __be32 key)
-{
- struct hlist_head *head = rtnl_dereference(fz->fz_hash) + fn_hash(key, fz);
- struct hlist_node *node;
- struct fib_node *f;
-
- hlist_for_each_entry_rcu(f, node, head, fn_hash) {
- if (f->fn_key == key)
- return f;
- }
-
- return NULL;
-}
-
-
-static struct fib_alias *fib_fast_alloc(struct fib_node *f)
-{
- struct fib_alias *fa = &f->fn_embedded_alias;
-
- if (fa->fa_info != NULL)
- fa = kmem_cache_alloc(fn_alias_kmem, GFP_KERNEL);
- return fa;
-}
-
-/* Caller must hold RTNL. */
-int fib_table_insert(struct fib_table *tb, struct fib_config *cfg)
-{
- struct fn_hash *table = (struct fn_hash *) tb->tb_data;
- struct fib_node *new_f = NULL;
- struct fib_node *f;
- struct fib_alias *fa, *new_fa;
- struct fn_zone *fz;
- struct fib_info *fi;
- u8 tos = cfg->fc_tos;
- __be32 key;
- int err;
-
- if (cfg->fc_dst_len > 32)
- return -EINVAL;
-
- fz = table->fn_zones[cfg->fc_dst_len];
- if (!fz && !(fz = fn_new_zone(table, cfg->fc_dst_len)))
- return -ENOBUFS;
-
- key = 0;
- if (cfg->fc_dst) {
- if (cfg->fc_dst & ~FZ_MASK(fz))
- return -EINVAL;
- key = fz_key(cfg->fc_dst, fz);
- }
-
- fi = fib_create_info(cfg);
- if (IS_ERR(fi))
- return PTR_ERR(fi);
-
- if (fz->fz_nent > (fz->fz_divisor<<1) &&
- fz->fz_divisor < FZ_MAX_DIVISOR &&
- (cfg->fc_dst_len == 32 ||
- (1 << cfg->fc_dst_len) > fz->fz_divisor))
- fn_rehash_zone(fz);
-
- f = fib_find_node(fz, key);
-
- if (!f)
- fa = NULL;
- else
- fa = fib_find_alias(&f->fn_alias, tos, fi->fib_priority);
-
- /* Now fa, if non-NULL, points to the first fib alias
- * with the same keys [prefix,tos,priority], if such key already
- * exists or to the node before which we will insert new one.
- *
- * If fa is NULL, we will need to allocate a new one and
- * insert to the head of f.
- *
- * If f is NULL, no fib node matched the destination key
- * and we need to allocate a new one of those as well.
- */
-
- if (fa && fa->fa_tos == tos &&
- fa->fa_info->fib_priority == fi->fib_priority) {
- struct fib_alias *fa_first, *fa_match;
-
- err = -EEXIST;
- if (cfg->fc_nlflags & NLM_F_EXCL)
- goto out;
-
- /* We have 2 goals:
- * 1. Find exact match for type, scope, fib_info to avoid
- * duplicate routes
- * 2. Find next 'fa' (or head), NLM_F_APPEND inserts before it
- */
- fa_match = NULL;
- fa_first = fa;
- fa = list_entry(fa->fa_list.prev, struct fib_alias, fa_list);
- list_for_each_entry_continue(fa, &f->fn_alias, fa_list) {
- if (fa->fa_tos != tos)
- break;
- if (fa->fa_info->fib_priority != fi->fib_priority)
- break;
- if (fa->fa_type == cfg->fc_type &&
- fa->fa_scope == cfg->fc_scope &&
- fa->fa_info == fi) {
- fa_match = fa;
- break;
- }
- }
-
- if (cfg->fc_nlflags & NLM_F_REPLACE) {
- u8 state;
-
- fa = fa_first;
- if (fa_match) {
- if (fa == fa_match)
- err = 0;
- goto out;
- }
- err = -ENOBUFS;
- new_fa = fib_fast_alloc(f);
- if (new_fa == NULL)
- goto out;
-
- new_fa->fa_tos = fa->fa_tos;
- new_fa->fa_info = fi;
- new_fa->fa_type = cfg->fc_type;
- new_fa->fa_scope = cfg->fc_scope;
- state = fa->fa_state;
- new_fa->fa_state = state & ~FA_S_ACCESSED;
- fib_hash_genid++;
- list_replace_rcu(&fa->fa_list, &new_fa->fa_list);
-
- fn_free_alias(fa, f);
- if (state & FA_S_ACCESSED)
- rt_cache_flush(cfg->fc_nlinfo.nl_net, -1);
- rtmsg_fib(RTM_NEWROUTE, key, new_fa, cfg->fc_dst_len,
- tb->tb_id, &cfg->fc_nlinfo, NLM_F_REPLACE);
- return 0;
- }
-
- /* Error if we find a perfect match which
- * uses the same scope, type, and nexthop
- * information.
- */
- if (fa_match)
- goto out;
-
- if (!(cfg->fc_nlflags & NLM_F_APPEND))
- fa = fa_first;
- }
-
- err = -ENOENT;
- if (!(cfg->fc_nlflags & NLM_F_CREATE))
- goto out;
-
- err = -ENOBUFS;
-
- if (!f) {
- new_f = kmem_cache_zalloc(fn_hash_kmem, GFP_KERNEL);
- if (new_f == NULL)
- goto out;
-
- INIT_HLIST_NODE(&new_f->fn_hash);
- INIT_LIST_HEAD(&new_f->fn_alias);
- new_f->fn_key = key;
- f = new_f;
- }
-
- new_fa = fib_fast_alloc(f);
- if (new_fa == NULL)
- goto out;
-
- new_fa->fa_info = fi;
- new_fa->fa_tos = tos;
- new_fa->fa_type = cfg->fc_type;
- new_fa->fa_scope = cfg->fc_scope;
- new_fa->fa_state = 0;
-
- /*
- * Insert new entry to the list.
- */
-
- if (new_f)
- fib_insert_node(fz, new_f);
- list_add_tail_rcu(&new_fa->fa_list,
- (fa ? &fa->fa_list : &f->fn_alias));
- fib_hash_genid++;
-
- if (new_f)
- fz->fz_nent++;
- rt_cache_flush(cfg->fc_nlinfo.nl_net, -1);
-
- rtmsg_fib(RTM_NEWROUTE, key, new_fa, cfg->fc_dst_len, tb->tb_id,
- &cfg->fc_nlinfo, 0);
- return 0;
-
-out:
- if (new_f)
- kmem_cache_free(fn_hash_kmem, new_f);
- fib_release_info(fi);
- return err;
-}
-
-int fib_table_delete(struct fib_table *tb, struct fib_config *cfg)
-{
- struct fn_hash *table = (struct fn_hash *)tb->tb_data;
- struct fib_node *f;
- struct fib_alias *fa, *fa_to_delete;
- struct fn_zone *fz;
- __be32 key;
-
- if (cfg->fc_dst_len > 32)
- return -EINVAL;
-
- if ((fz = table->fn_zones[cfg->fc_dst_len]) == NULL)
- return -ESRCH;
-
- key = 0;
- if (cfg->fc_dst) {
- if (cfg->fc_dst & ~FZ_MASK(fz))
- return -EINVAL;
- key = fz_key(cfg->fc_dst, fz);
- }
-
- f = fib_find_node(fz, key);
-
- if (!f)
- fa = NULL;
- else
- fa = fib_find_alias(&f->fn_alias, cfg->fc_tos, 0);
- if (!fa)
- return -ESRCH;
-
- fa_to_delete = NULL;
- fa = list_entry(fa->fa_list.prev, struct fib_alias, fa_list);
- list_for_each_entry_continue(fa, &f->fn_alias, fa_list) {
- struct fib_info *fi = fa->fa_info;
-
- if (fa->fa_tos != cfg->fc_tos)
- break;
-
- if ((!cfg->fc_type ||
- fa->fa_type == cfg->fc_type) &&
- (cfg->fc_scope == RT_SCOPE_NOWHERE ||
- fa->fa_scope == cfg->fc_scope) &&
- (!cfg->fc_protocol ||
- fi->fib_protocol == cfg->fc_protocol) &&
- fib_nh_match(cfg, fi) == 0) {
- fa_to_delete = fa;
- break;
- }
- }
-
- if (fa_to_delete) {
- int kill_fn;
-
- fa = fa_to_delete;
- rtmsg_fib(RTM_DELROUTE, key, fa, cfg->fc_dst_len,
- tb->tb_id, &cfg->fc_nlinfo, 0);
-
- kill_fn = 0;
- list_del_rcu(&fa->fa_list);
- if (list_empty(&f->fn_alias)) {
- hlist_del_rcu(&f->fn_hash);
- kill_fn = 1;
- }
- fib_hash_genid++;
-
- if (fa->fa_state & FA_S_ACCESSED)
- rt_cache_flush(cfg->fc_nlinfo.nl_net, -1);
- fn_free_alias(fa, f);
- if (kill_fn) {
- fn_free_node(f);
- fz->fz_nent--;
- }
-
- return 0;
- }
- return -ESRCH;
-}
-
-static int fn_flush_list(struct fn_zone *fz, int idx)
-{
- struct hlist_head *head = rtnl_dereference(fz->fz_hash) + idx;
- struct hlist_node *node, *n;
- struct fib_node *f;
- int found = 0;
-
- hlist_for_each_entry_safe(f, node, n, head, fn_hash) {
- struct fib_alias *fa, *fa_node;
- int kill_f;
-
- kill_f = 0;
- list_for_each_entry_safe(fa, fa_node, &f->fn_alias, fa_list) {
- struct fib_info *fi = fa->fa_info;
-
- if (fi && (fi->fib_flags&RTNH_F_DEAD)) {
- list_del_rcu(&fa->fa_list);
- if (list_empty(&f->fn_alias)) {
- hlist_del_rcu(&f->fn_hash);
- kill_f = 1;
- }
- fib_hash_genid++;
-
- fn_free_alias(fa, f);
- found++;
- }
- }
- if (kill_f) {
- fn_free_node(f);
- fz->fz_nent--;
- }
- }
- return found;
-}
-
-/* caller must hold RTNL. */
-int fib_table_flush(struct fib_table *tb)
-{
- struct fn_hash *table = (struct fn_hash *) tb->tb_data;
- struct fn_zone *fz;
- int found = 0;
-
- for (fz = rtnl_dereference(table->fn_zone_list);
- fz != NULL;
- fz = rtnl_dereference(fz->fz_next)) {
- int i;
-
- for (i = fz->fz_divisor - 1; i >= 0; i--)
- found += fn_flush_list(fz, i);
- }
- return found;
-}
-
-void fib_free_table(struct fib_table *tb)
-{
- struct fn_hash *table = (struct fn_hash *) tb->tb_data;
- struct fn_zone *fz, *next;
-
- next = table->fn_zone_list;
- while (next != NULL) {
- fz = next;
- next = fz->fz_next;
-
- if (fz->fz_hash != fz->fz_embedded_hash)
- fz_hash_free(fz->fz_hash, fz->fz_divisor);
-
- kfree(fz);
- }
-
- kfree(tb);
-}
-
-static inline int
-fn_hash_dump_bucket(struct sk_buff *skb, struct netlink_callback *cb,
- struct fib_table *tb,
- struct fn_zone *fz,
- struct hlist_head *head)
-{
- struct hlist_node *node;
- struct fib_node *f;
- int i, s_i;
-
- s_i = cb->args[4];
- i = 0;
- hlist_for_each_entry_rcu(f, node, head, fn_hash) {
- struct fib_alias *fa;
-
- list_for_each_entry_rcu(fa, &f->fn_alias, fa_list) {
- if (i < s_i)
- goto next;
-
- if (fib_dump_info(skb, NETLINK_CB(cb->skb).pid,
- cb->nlh->nlmsg_seq,
- RTM_NEWROUTE,
- tb->tb_id,
- fa->fa_type,
- fa->fa_scope,
- f->fn_key,
- fz->fz_order,
- fa->fa_tos,
- fa->fa_info,
- NLM_F_MULTI) < 0) {
- cb->args[4] = i;
- return -1;
- }
-next:
- i++;
- }
- }
- cb->args[4] = i;
- return skb->len;
-}
-
-static inline int
-fn_hash_dump_zone(struct sk_buff *skb, struct netlink_callback *cb,
- struct fib_table *tb,
- struct fn_zone *fz)
-{
- int h, s_h;
- struct hlist_head *head = rcu_dereference(fz->fz_hash);
-
- if (head == NULL)
- return skb->len;
- s_h = cb->args[3];
- for (h = s_h; h < fz->fz_divisor; h++) {
- if (hlist_empty(head + h))
- continue;
- if (fn_hash_dump_bucket(skb, cb, tb, fz, head + h) < 0) {
- cb->args[3] = h;
- return -1;
- }
- memset(&cb->args[4], 0,
- sizeof(cb->args) - 4*sizeof(cb->args[0]));
- }
- cb->args[3] = h;
- return skb->len;
-}
-
-int fib_table_dump(struct fib_table *tb, struct sk_buff *skb,
- struct netlink_callback *cb)
-{
- int m = 0, s_m;
- struct fn_zone *fz;
- struct fn_hash *table = (struct fn_hash *)tb->tb_data;
-
- s_m = cb->args[2];
- rcu_read_lock();
- for (fz = rcu_dereference(table->fn_zone_list);
- fz != NULL;
- fz = rcu_dereference(fz->fz_next), m++) {
- if (m < s_m)
- continue;
- if (fn_hash_dump_zone(skb, cb, tb, fz) < 0) {
- cb->args[2] = m;
- rcu_read_unlock();
- return -1;
- }
- memset(&cb->args[3], 0,
- sizeof(cb->args) - 3*sizeof(cb->args[0]));
- }
- rcu_read_unlock();
- cb->args[2] = m;
- return skb->len;
-}
-
-void __init fib_hash_init(void)
-{
- fn_hash_kmem = kmem_cache_create("ip_fib_hash", sizeof(struct fib_node),
- 0, SLAB_PANIC, NULL);
-
- fn_alias_kmem = kmem_cache_create("ip_fib_alias", sizeof(struct fib_alias),
- 0, SLAB_PANIC, NULL);
-
-}
-
-struct fib_table *fib_hash_table(u32 id)
-{
- struct fib_table *tb;
-
- tb = kmalloc(sizeof(struct fib_table) + sizeof(struct fn_hash),
- GFP_KERNEL);
- if (tb == NULL)
- return NULL;
-
- tb->tb_id = id;
- tb->tb_default = -1;
-
- memset(tb->tb_data, 0, sizeof(struct fn_hash));
- return tb;
-}
-
-/* ------------------------------------------------------------------------ */
-#ifdef CONFIG_PROC_FS
-
-struct fib_iter_state {
- struct seq_net_private p;
- struct fn_zone *zone;
- int bucket;
- struct hlist_head *hash_head;
- struct fib_node *fn;
- struct fib_alias *fa;
- loff_t pos;
- unsigned int genid;
- int valid;
-};
-
-static struct fib_alias *fib_get_first(struct seq_file *seq)
-{
- struct fib_iter_state *iter = seq->private;
- struct fib_table *main_table;
- struct fn_hash *table;
-
- main_table = fib_get_table(seq_file_net(seq), RT_TABLE_MAIN);
- table = (struct fn_hash *)main_table->tb_data;
-
- iter->bucket = 0;
- iter->hash_head = NULL;
- iter->fn = NULL;
- iter->fa = NULL;
- iter->pos = 0;
- iter->genid = fib_hash_genid;
- iter->valid = 1;
-
- for (iter->zone = rcu_dereference(table->fn_zone_list);
- iter->zone != NULL;
- iter->zone = rcu_dereference(iter->zone->fz_next)) {
- int maxslot;
-
- if (!iter->zone->fz_nent)
- continue;
-
- iter->hash_head = rcu_dereference(iter->zone->fz_hash);
- maxslot = iter->zone->fz_divisor;
-
- for (iter->bucket = 0; iter->bucket < maxslot;
- ++iter->bucket, ++iter->hash_head) {
- struct hlist_node *node;
- struct fib_node *fn;
-
- hlist_for_each_entry(fn, node, iter->hash_head, fn_hash) {
- struct fib_alias *fa;
-
- list_for_each_entry(fa, &fn->fn_alias, fa_list) {
- iter->fn = fn;
- iter->fa = fa;
- goto out;
- }
- }
- }
- }
-out:
- return iter->fa;
-}
-
-static struct fib_alias *fib_get_next(struct seq_file *seq)
-{
- struct fib_iter_state *iter = seq->private;
- struct fib_node *fn;
- struct fib_alias *fa;
-
- /* Advance FA, if any. */
- fn = iter->fn;
- fa = iter->fa;
- if (fa) {
- BUG_ON(!fn);
- list_for_each_entry_continue(fa, &fn->fn_alias, fa_list) {
- iter->fa = fa;
- goto out;
- }
- }
-
- fa = iter->fa = NULL;
-
- /* Advance FN. */
- if (fn) {
- struct hlist_node *node = &fn->fn_hash;
- hlist_for_each_entry_continue(fn, node, fn_hash) {
- iter->fn = fn;
-
- list_for_each_entry(fa, &fn->fn_alias, fa_list) {
- iter->fa = fa;
- goto out;
- }
- }
- }
-
- fn = iter->fn = NULL;
-
- /* Advance hash chain. */
- if (!iter->zone)
- goto out;
-
- for (;;) {
- struct hlist_node *node;
- int maxslot;
-
- maxslot = iter->zone->fz_divisor;
-
- while (++iter->bucket < maxslot) {
- iter->hash_head++;
-
- hlist_for_each_entry(fn, node, iter->hash_head, fn_hash) {
- list_for_each_entry(fa, &fn->fn_alias, fa_list) {
- iter->fn = fn;
- iter->fa = fa;
- goto out;
- }
- }
- }
-
- iter->zone = rcu_dereference(iter->zone->fz_next);
-
- if (!iter->zone)
- goto out;
-
- iter->bucket = 0;
- iter->hash_head = rcu_dereference(iter->zone->fz_hash);
-
- hlist_for_each_entry(fn, node, iter->hash_head, fn_hash) {
- list_for_each_entry(fa, &fn->fn_alias, fa_list) {
- iter->fn = fn;
- iter->fa = fa;
- goto out;
- }
- }
- }
-out:
- iter->pos++;
- return fa;
-}
-
-static struct fib_alias *fib_get_idx(struct seq_file *seq, loff_t pos)
-{
- struct fib_iter_state *iter = seq->private;
- struct fib_alias *fa;
-
- if (iter->valid && pos >= iter->pos && iter->genid == fib_hash_genid) {
- fa = iter->fa;
- pos -= iter->pos;
- } else
- fa = fib_get_first(seq);
-
- if (fa)
- while (pos && (fa = fib_get_next(seq)))
- --pos;
- return pos ? NULL : fa;
-}
-
-static void *fib_seq_start(struct seq_file *seq, loff_t *pos)
- __acquires(RCU)
-{
- void *v = NULL;
-
- rcu_read_lock();
- if (fib_get_table(seq_file_net(seq), RT_TABLE_MAIN))
- v = *pos ? fib_get_idx(seq, *pos - 1) : SEQ_START_TOKEN;
- return v;
-}
-
-static void *fib_seq_next(struct seq_file *seq, void *v, loff_t *pos)
-{
- ++*pos;
- return v == SEQ_START_TOKEN ? fib_get_first(seq) : fib_get_next(seq);
-}
-
-static void fib_seq_stop(struct seq_file *seq, void *v)
- __releases(RCU)
-{
- rcu_read_unlock();
-}
-
-static unsigned fib_flag_trans(int type, __be32 mask, struct fib_info *fi)
-{
- static const unsigned type2flags[RTN_MAX + 1] = {
- [7] = RTF_REJECT,
- [8] = RTF_REJECT,
- };
- unsigned flags = type2flags[type];
-
- if (fi && fi->fib_nh->nh_gw)
- flags |= RTF_GATEWAY;
- if (mask == htonl(0xFFFFFFFF))
- flags |= RTF_HOST;
- flags |= RTF_UP;
- return flags;
-}
-
-/*
- * This outputs /proc/net/route.
- *
- * It always works in backward compatibility mode.
- * The format of the file is not supposed to be changed.
- */
-static int fib_seq_show(struct seq_file *seq, void *v)
-{
- struct fib_iter_state *iter;
- int len;
- __be32 prefix, mask;
- unsigned flags;
- struct fib_node *f;
- struct fib_alias *fa;
- struct fib_info *fi;
-
- if (v == SEQ_START_TOKEN) {
- seq_printf(seq, "%-127s\n", "Iface\tDestination\tGateway "
- "\tFlags\tRefCnt\tUse\tMetric\tMask\t\tMTU"
- "\tWindow\tIRTT");
- goto out;
- }
-
- iter = seq->private;
- f = iter->fn;
- fa = iter->fa;
- fi = fa->fa_info;
- prefix = f->fn_key;
- mask = FZ_MASK(iter->zone);
- flags = fib_flag_trans(fa->fa_type, mask, fi);
- if (fi)
- seq_printf(seq,
- "%s\t%08X\t%08X\t%04X\t%d\t%u\t%d\t%08X\t%d\t%u\t%u%n",
- fi->fib_dev ? fi->fib_dev->name : "*", prefix,
- fi->fib_nh->nh_gw, flags, 0, 0, fi->fib_priority,
- mask, (fi->fib_advmss ? fi->fib_advmss + 40 : 0),
- fi->fib_window,
- fi->fib_rtt >> 3, &len);
- else
- seq_printf(seq,
- "*\t%08X\t%08X\t%04X\t%d\t%u\t%d\t%08X\t%d\t%u\t%u%n",
- prefix, 0, flags, 0, 0, 0, mask, 0, 0, 0, &len);
-
- seq_printf(seq, "%*s\n", 127 - len, "");
-out:
- return 0;
-}
-
-static const struct seq_operations fib_seq_ops = {
- .start = fib_seq_start,
- .next = fib_seq_next,
- .stop = fib_seq_stop,
- .show = fib_seq_show,
-};
-
-static int fib_seq_open(struct inode *inode, struct file *file)
-{
- return seq_open_net(inode, file, &fib_seq_ops,
- sizeof(struct fib_iter_state));
-}
-
-static const struct file_operations fib_seq_fops = {
- .owner = THIS_MODULE,
- .open = fib_seq_open,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = seq_release_net,
-};
-
-int __net_init fib_proc_init(struct net *net)
-{
- if (!proc_net_fops_create(net, "route", S_IRUGO, &fib_seq_fops))
- return -ENOMEM;
- return 0;
-}
-
-void __net_exit fib_proc_exit(struct net *net)
-{
- proc_net_remove(net, "route");
-}
-#endif /* CONFIG_PROC_FS */
--
1.7.4
^ permalink raw reply related
* Re: [PATCH 2/2] eCryptfs: lock requested keys for writing
From: Tyler Hicks @ 2011-02-01 23:16 UTC (permalink / raw)
To: Roberto Sassu; +Cc: kirkland, dhowells, keyrings, linux-fsdevel, linux-kernel
In-Reply-To: <1296211326-2437-2-git-send-email-roberto.sassu@polito.it>
On Fri Jan 28, 2011 at 11:42:04AM +0100, Roberto Sassu <roberto.sassu@polito.it> wrote:
> Keys requested by eCryptfs are exclusively locked in order to prevent
> modifications by other subjects. In particular those which description is
> specified at mount time are locked until the filesystem is unmounted, these
I don't think the approach taken for the mount wide keys will work for a
couple reasons. The first is that lockdep reports this after a mount:
================================================
[ BUG: lock held when returning to user space! ]
------------------------------------------------
mount/1019 is leaving the kernel with locks still held!
1 lock held by mount/1019:
#0: (&key->sem){+.+.+.}, at: [<ffffffffa021ff12>] ecryptfs_keyring_auth_tok_for_sig+0xe6/0x195 [ecryptfs]
The second is that while this does prevent key_update() from updating
the key underneath us, it just results in `keyctl update` on a mount key
to become a hung task until unmounting the eCryptfs mount.
It looks like we'll have to have more efficient locking for mount keys.
Tyler
> required to decrypt a single file are unlocked immediately after the open.
>
> Signed-off-by: Roberto Sassu <roberto.sassu@polito.it>
> Reported-by: David Howells <dhowells@redhat.com>
> ---
> fs/ecryptfs/crypto.c | 4 +++-
> fs/ecryptfs/keystore.c | 15 ++++++++++++---
> 2 files changed, 15 insertions(+), 4 deletions(-)
>
> diff --git a/fs/ecryptfs/crypto.c b/fs/ecryptfs/crypto.c
> index bfd8b68..6b32776 100644
> --- a/fs/ecryptfs/crypto.c
> +++ b/fs/ecryptfs/crypto.c
> @@ -268,8 +268,10 @@ void ecryptfs_destroy_mount_crypt_stat(
> list_del(&auth_tok->mount_crypt_stat_list);
> mount_crypt_stat->num_global_auth_toks--;
> if (auth_tok->global_auth_tok_key
> - && !(auth_tok->flags & ECRYPTFS_AUTH_TOK_INVALID))
> + && !(auth_tok->flags & ECRYPTFS_AUTH_TOK_INVALID)) {
> + up_write(&(auth_tok->global_auth_tok_key->sem));
> key_put(auth_tok->global_auth_tok_key);
> + }
> kmem_cache_free(ecryptfs_global_auth_tok_cache, auth_tok);
> }
> mutex_unlock(&mount_crypt_stat->global_auth_tok_list_mutex);
> diff --git a/fs/ecryptfs/keystore.c b/fs/ecryptfs/keystore.c
> index 4feb78c..c90456a 100644
> --- a/fs/ecryptfs/keystore.c
> +++ b/fs/ecryptfs/keystore.c
> @@ -765,8 +765,10 @@ out_free_unlock:
> out_unlock:
> mutex_unlock(s->tfm_mutex);
> out:
> - if (auth_tok_key)
> + if (auth_tok_key) {
> + up_write(&(auth_tok_key->sem));
> key_put(auth_tok_key);
> + }
> kfree(s);
> return rc;
> }
> @@ -1002,8 +1004,10 @@ out:
> (*filename_size) = 0;
> (*filename) = NULL;
> }
> - if (auth_tok_key)
> + if (auth_tok_key) {
> + up_write(&(auth_tok_key->sem));
> key_put(auth_tok_key);
> + }
> kfree(s);
> return rc;
> }
> @@ -1566,6 +1570,7 @@ int ecryptfs_keyring_auth_tok_for_sig(struct key **auth_tok_key,
> (*auth_tok_key) = NULL;
> goto out;
> }
> + down_write(&((*auth_tok_key)->sem));
> (*auth_tok) = ecryptfs_get_key_payload_data(*auth_tok_key);
> if (ecryptfs_verify_version((*auth_tok)->version)) {
> printk(KERN_ERR
> @@ -1587,6 +1592,7 @@ int ecryptfs_keyring_auth_tok_for_sig(struct key **auth_tok_key,
> }
> out_release_key:
> if (rc) {
> + up_write(&((*auth_tok_key)->sem));
> key_put(*auth_tok_key);
> (*auth_tok_key) = NULL;
> }
> @@ -1810,6 +1816,7 @@ int ecryptfs_parse_packet_set(struct ecryptfs_crypt_stat *crypt_stat,
> find_next_matching_auth_tok:
> found_auth_tok = 0;
> if (auth_tok_key) {
> + up_write(&(auth_tok_key->sem));
> key_put(auth_tok_key);
> auth_tok_key = NULL;
> }
> @@ -1896,8 +1903,10 @@ found_matching_auth_tok:
> out_wipe_list:
> wipe_auth_tok_list(&auth_tok_list);
> out:
> - if (auth_tok_key)
> + if (auth_tok_key) {
> + up_write(&(auth_tok_key->sem));
> key_put(auth_tok_key);
> + }
> return rc;
> }
>
> --
> 1.7.3.4
>
^ permalink raw reply
* Policy on (committing) large (1MB) machine specific patches?
From: Leon Woestenberg @ 2011-02-01 23:15 UTC (permalink / raw)
To: openembedded-devel
Hello,
I would like to submit another machine to OE, but I need to bring in
vendor patches (which do not apply cleanly) modified by me, so they
have no Internet presence.
The patches measure in 1MB of plain source text.
Is it policy to commit this? Where is the limit?
I see we have lots of large patches:
cd openembedded.git/recipes/linux/
ls -Sshr `find . -type f -regex .*patch$`
...many of them appearing multiple times in the checkout:
ls -Shr `find . -type f -regex .*patch$` | xargs -L1 md5sum
eating disk space on the build machine.
Regards,
--
Leon
^ permalink raw reply
* Re: [1.8.0] Remote tag namespace
From: Johan Herland @ 2011-02-01 23:15 UTC (permalink / raw)
To: Jeff King; +Cc: git, Nguyen Thai Ngoc Duy, Nicolas Pitre
In-Reply-To: <20110201181428.GA6579@sigill.intra.peff.net>
On Tuesday 01 February 2011, Jeff King wrote:
> On Tue, Feb 01, 2011 at 05:44:50PM +0700, Nguyen Thai Ngoc Duy wrote:
> > OK I'm not familiar with tag code, but I can try.
> >
> > Proposal:
> >
> > Reserve refs/remote-tags namespace to store tags from remotes. Its
> > structure is the same as in refs/remotes. When pulling tags, put them
> > in refs/remote-tags/<remote> instead of refs/tags.
> > Tag dereference code will be taught about refs/remote-tags with
> > similar deref order as in remote branches.
>
> There are similar questions around remote notes refs. Should there also
> be a refs/remote-notes? And there was some discussion recently about
> fetching remote replace refs.
>
> Should we perhaps be massaging refs/remotes into a structure to handle
> all of these things? Like:
>
> refs/remotes/origin/HEAD (-> refs/remotes/origin/heads/master)
> refs/remotes/origin/heads/master
> refs/remotes/origin/tags/v1.7.4
> refs/remotes/origin/notes/commit
> refs/remotes/origin/replace/f67e92af477a2255b64a1ece33d9d126e763fe9b
>
> i.e., make refs/remotes/* an actual mirror of selected parts of the
> remote's refs/ hierarchy. And then figure out sane rules for merging
> those namespaces into the ref lookup procedure. For heads and tags,
> probably some tweaking of the lookup rules in dwim_ref; for
> replacements, probably you would want to manually say "I am interested
> in this replace" and copy or symref-link it into your refs/ hierarchy.
I fully agree.
In addition - as discussed in http://thread.gmane.org/gmane.comp.version-
control.git/160503/focus=160795 - we should also tweak the refspec format to
make tag auto-following explicit in the refspec.
> And probably something similar with notes.
(going slightly offtopic with the notes discussion here)
I've been thinking that notes should be organized much in the same fashion
as branches/heads. There should be remote notes refs that should only be
updated from the remote, and there should be local notes refs in
refs/notes/*. You should be able to configure upstream relationships between
local notes refs and remote notes refs (e.g. notes.foo.remote and
notes.foo.merge), and auto-merge them on "git notes pull".
Have fun! :)
...Johan
--
Johan Herland, <johan@herland.net>
www.herland.net
^ permalink raw reply
* Re: [1.8.0] Remote tag namespace
From: Sverre Rabbelier @ 2011-02-01 23:14 UTC (permalink / raw)
To: Jeff King; +Cc: Nguyen Thai Ngoc Duy, Git Mailing List, Nicolas Pitre
In-Reply-To: <20110201181428.GA6579@sigill.intra.peff.net>
Heya,
On Tue, Feb 1, 2011 at 19:14, Jeff King <peff@peff.net> wrote:
> i.e., make refs/remotes/* an actual mirror of selected parts of the
> remote's refs/ hierarchy. And then figure out sane rules for merging
> those namespaces into the ref lookup procedure.
Jeff, Nguy, are either of you interested in writing up a new/modifying
this proposal to be about namespacing everything? I think it
definitely makes sense to have a namespace for notes, replaces, as
well as tags, especially since it would also allow us to propagate
these by default (at least to the refs/remotes namespace), and think
it's a good idea to go all the way if we're going to do this at all
(or we'll have the same discussion again later for notes and replaces,
and whatever comes after that).
--
Cheers,
Sverre Rabbelier
^ permalink raw reply
* Re: [PATCH 0/2] x86/microcode: support for microcode update in Xen dom0
From: Jeremy Fitzhardinge @ 2011-02-01 23:12 UTC (permalink / raw)
To: Borislav Petkov, H. Peter Anvin, Ingo Molnar,
the arch/x86 maintainers, Linux Kernel Mailing List, Xen Devel,
Jeremy Fitzhardinge
In-Reply-To: <20110201110026.GA4739@liondog.tnic>
On 02/01/2011 03:00 AM, Borislav Petkov wrote:
> I am thinking something in the sense of the above. For example, in the
> AMD case you take
>
> static struct microcode_ops microcode_amd_ops = {
> .request_microcode_user = request_microcode_user,
> .request_microcode_fw = request_microcode_fw,
> .collect_cpu_info = collect_cpu_info_amd,
> .apply_microcode = apply_microcode_amd,
> .microcode_fini_cpu = microcode_fini_cpu_amd,
> };
>
> and reuse the ->request_microcode_fw, ->collect_cpu_info and
> ->microcode_fini_cpu on dom0 as if you're running on baremetal. Up
> to the point where you need to apply the microcode. Then, you use
> your supplied ->apply_microcode hypercall wrapper to call into the
> hypervisor.
collect_cpu_info can't work, because the domain doesn't have access to
all the host's physical CPUs.
However, even aside from that, it means exporting a pile of internal
details from microcode_amd and reusing them within microcode_xen. And
it requires that it be done again for each vendor.
But all that's really needed is a dead simple "request" that loads the
entire file (with a vendor-specific name) and shoves it into Xen.
There's no need for any vendor-specific code beyond the filename.
>> But all this is flawed because the microcode_intel/amd.c drivers assume
>> they can see all the CPUs present in the system, and load suitable
>> microcode for each specific one. But a kernel in a Xen domain only has
>> virtual CPUs - not physical ones - and has no idea how to get
>> appropriate microcode data for all the physical CPUs in the system.
> Well, let me quote you:
>
> On Fri, Jan 28, 2011 at 04:26:52PM -0800, Jeremy Fitzhardinge wrote:
>> Xen update mechanism is uniform independent of the CPU type, but the
>> driver must know where to find the data file, which depends on the CPU
>> type. And since the update hypercall updates all CPUs, we only need to
>> execute it once on any CPU - but for simplicity it just runs it only
>> on (V)CPU 0.
> so you only do it once and exit early in the rest of the cases. I
> wouldn't worry about performance since ucode is applied only once upon
> boot.
Its not a performance question. The Intel and AMD microcode drivers
parse the full blob loaded from userspace, and just extract the chunk
needed for each CPU. It does this for each separate CPU, so in
principle you could have a mixture of models within one machine or
something (the driver certainly assumes that could happen; perhaps it
could on a larger multinode machine).
The point is that if it does this on (what the domain sees as ) "cpu 0",
then it may throw away microcode chunks needed for other CPUs. That's
why we need to hand Xen the entire microcode file, and let the
hypervisor do the work of splitting it up and installing it on the CPUs.
> This is exactly what I'm talking about - why copy all that
> checking/filtering code from baremetal to Xen instead of simply reusing
> it? Especially if you'd need to update the copy from time to time when
> baremetal changes.
The code in the kernel is in the wrong place. It has to be done in
Xen. When Xen is present, the code in the kernel is redundant, not the
other way around.
>> CPU vendors test Xen, and Intel is particularly interested in getting
>> this microcode driver upstream. The amount of duplicated code is
>> trivial, and the basic structure of the microcode updates doesn't seem
>> set to change.
> Uuh, I wouldn't bet on that though :).
Shrug. AFAICT the mechanism hasn't changed since it was first
introduced. If there's a change, then both Linux and Xen will have to
change, and most likely the same CPU vendor engineer will provide a
patch for both. Xen has a good record for tracking new CPU features.
>> Since Xen has to have all sorts of other CPU-specific code which at
>> least somewhat overlaps with what's in the kernel a bit more doesn't
>> matter.
> Well, I'll let x86 people decide on that but last time I checked they
> opposed "if (xen)" sprinkling all over the place.
Eh? I'm talking about code within Xen; it doesn't involve any if (xen)s
within the kernel.
> Btw, hpa has a point, if you can load microcode using multiboot, all
> that discussion will become moot since you'll be better at loading
> microcode even than baremetal. We need a similar mechanism in x86 too
> since the current one loads the microcode definitely too late.
>
> The optimal case for baremetal would be to load it as early as possible
> on each CPU's bootstrapping path and if you can do that in the
> hypervisor, before even dom0 starts, you're very much fine.
It is possible, but it requires that vendors install the microcode
updates in /boot and update the grub entries accordingly. I'd prefer a
solution which works with current distros as-is.
J
^ permalink raw reply
* Re: What's the typical RAID10 setup?
From: Roberto Spadim @ 2011-02-01 23:12 UTC (permalink / raw)
To: Stan Hoeppner; +Cc: Jon Nelson, David Brown, linux-raid
In-Reply-To: <4D4883A3.6030605@hardwarefreak.com>
again....
closest head algorithm (today raid1)
is good for hard disks
but isn´t good for ssd (round robin here is better)
but the best algorithm is time based (minimize time to access data)
2011/2/1 Stan Hoeppner <stan@hardwarefreak.com>:
> Jon Nelson put forth on 2/1/2011 7:50 AM:
>
>> The performance will not be the same because. Whenever possible, md
>> reads from the outermost portion of the disk -- theoretically the
>> fastest portion of the disk (by 2 or 3 times as much as the inner
>> tracks) -- and in this way raid10,f2 can actually be faster than
>> raid0.
>
> Faster in what regard? I assume you mean purely sequential read, and not random
> IOPS. The access patterns of the vast majority of workloads are random, so I
> don't see much real world benefit, if what you say is correct. This might
> benefit MythTV or similar niche streaming apps.
>
> --
> Stan
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 0/2] x86/microcode: support for microcode update in Xen dom0
From: Jeremy Fitzhardinge @ 2011-02-01 23:12 UTC (permalink / raw)
To: Borislav Petkov, H. Peter Anvin, Ingo Molnar,
the arch/x86 maintainers, Linux Kernel Mailing List
In-Reply-To: <20110201110026.GA4739@liondog.tnic>
On 02/01/2011 03:00 AM, Borislav Petkov wrote:
> I am thinking something in the sense of the above. For example, in the
> AMD case you take
>
> static struct microcode_ops microcode_amd_ops = {
> .request_microcode_user = request_microcode_user,
> .request_microcode_fw = request_microcode_fw,
> .collect_cpu_info = collect_cpu_info_amd,
> .apply_microcode = apply_microcode_amd,
> .microcode_fini_cpu = microcode_fini_cpu_amd,
> };
>
> and reuse the ->request_microcode_fw, ->collect_cpu_info and
> ->microcode_fini_cpu on dom0 as if you're running on baremetal. Up
> to the point where you need to apply the microcode. Then, you use
> your supplied ->apply_microcode hypercall wrapper to call into the
> hypervisor.
collect_cpu_info can't work, because the domain doesn't have access to
all the host's physical CPUs.
However, even aside from that, it means exporting a pile of internal
details from microcode_amd and reusing them within microcode_xen. And
it requires that it be done again for each vendor.
But all that's really needed is a dead simple "request" that loads the
entire file (with a vendor-specific name) and shoves it into Xen.
There's no need for any vendor-specific code beyond the filename.
>> But all this is flawed because the microcode_intel/amd.c drivers assume
>> they can see all the CPUs present in the system, and load suitable
>> microcode for each specific one. But a kernel in a Xen domain only has
>> virtual CPUs - not physical ones - and has no idea how to get
>> appropriate microcode data for all the physical CPUs in the system.
> Well, let me quote you:
>
> On Fri, Jan 28, 2011 at 04:26:52PM -0800, Jeremy Fitzhardinge wrote:
>> Xen update mechanism is uniform independent of the CPU type, but the
>> driver must know where to find the data file, which depends on the CPU
>> type. And since the update hypercall updates all CPUs, we only need to
>> execute it once on any CPU - but for simplicity it just runs it only
>> on (V)CPU 0.
> so you only do it once and exit early in the rest of the cases. I
> wouldn't worry about performance since ucode is applied only once upon
> boot.
Its not a performance question. The Intel and AMD microcode drivers
parse the full blob loaded from userspace, and just extract the chunk
needed for each CPU. It does this for each separate CPU, so in
principle you could have a mixture of models within one machine or
something (the driver certainly assumes that could happen; perhaps it
could on a larger multinode machine).
The point is that if it does this on (what the domain sees as ) "cpu 0",
then it may throw away microcode chunks needed for other CPUs. That's
why we need to hand Xen the entire microcode file, and let the
hypervisor do the work of splitting it up and installing it on the CPUs.
> This is exactly what I'm talking about - why copy all that
> checking/filtering code from baremetal to Xen instead of simply reusing
> it? Especially if you'd need to update the copy from time to time when
> baremetal changes.
The code in the kernel is in the wrong place. It has to be done in
Xen. When Xen is present, the code in the kernel is redundant, not the
other way around.
>> CPU vendors test Xen, and Intel is particularly interested in getting
>> this microcode driver upstream. The amount of duplicated code is
>> trivial, and the basic structure of the microcode updates doesn't seem
>> set to change.
> Uuh, I wouldn't bet on that though :).
Shrug. AFAICT the mechanism hasn't changed since it was first
introduced. If there's a change, then both Linux and Xen will have to
change, and most likely the same CPU vendor engineer will provide a
patch for both. Xen has a good record for tracking new CPU features.
>> Since Xen has to have all sorts of other CPU-specific code which at
>> least somewhat overlaps with what's in the kernel a bit more doesn't
>> matter.
> Well, I'll let x86 people decide on that but last time I checked they
> opposed "if (xen)" sprinkling all over the place.
Eh? I'm talking about code within Xen; it doesn't involve any if (xen)s
within the kernel.
> Btw, hpa has a point, if you can load microcode using multiboot, all
> that discussion will become moot since you'll be better at loading
> microcode even than baremetal. We need a similar mechanism in x86 too
> since the current one loads the microcode definitely too late.
>
> The optimal case for baremetal would be to load it as early as possible
> on each CPU's bootstrapping path and if you can do that in the
> hypervisor, before even dom0 starts, you're very much fine.
It is possible, but it requires that vendors install the microcode
updates in /boot and update the grub entries accordingly. I'd prefer a
solution which works with current distros as-is.
J
^ permalink raw reply
* Re: [PATCHv4] OMAP: Enable Magic SysRq on serial console ttyOx
From: Kevin Hilman @ 2011-02-01 23:10 UTC (permalink / raw)
To: Greg KH; +Cc: linux-omap, Govindraj.R, open list, Thomas Weber
In-Reply-To: <20110201222342.GA19137@suse.de>
Greg KH <gregkh@suse.de> writes:
> On Tue, Feb 01, 2011 at 08:31:30AM -0800, Kevin Hilman wrote:
>> Thomas Weber <weber@corscience.de> writes:
>>
>> > Magic SysRq key is not working for OMAP on new serial
>> > console ttyOx because SUPPORT_SYSRQ is not defined
>> > for omap-serial.
>> >
>> > This patch defines SUPPORT_SYSRQ in omap-serial and
>> > enables handling of Magic SysRq character.
>> >
>> > Further there is an issue of losing first break character.
>> > Removing the reset of the lsr_break_flag fixes this issue.
>> >
>> > Signed-off-by: Thomas Weber <weber@corscience.de>
>> > Acked-by: Govindraj.R <govindraj.raja@ti.com>
>> > Tested-by: Manjunath G Kondaiah <manjugk@ti.com>
>> > Acked-by: Kevin Hilman <khilman@ti.com>
>>
>> Greg,
>>
>> Will you be taking this? If you prefer, we can take it via OMAP tree
>> with your ack.
>
> I can take it if it's not a problem for you. Or if you want to, please
> add:
> Acked-by: Greg Kroah-Hartman <gregkh@suse.de>
> to the patch.
>
> Let me know which is easier for you.
Since there are no dependencies on arch/arm/*omap*, you can merge it.
Thanks,
Kevin
^ permalink raw reply
* Re: [PATCHv2 dontapply] vhost-net tx tuning
From: Sridhar Samudrala @ 2011-02-01 23:07 UTC (permalink / raw)
To: Michael S. Tsirkin; +Cc: Steve Dobbelstein, mashirle, kvm, netdev
In-Reply-To: <20110201155253.GA22959@redhat.com>
On Tue, 2011-02-01 at 17:52 +0200, Michael S. Tsirkin wrote:
> OK, so thinking about it more, maybe the issue is this:
> tx becomes full. We process one request and interrupt the guest,
> then it adds one request and the queue is full again.
>
> Maybe the following will help it stabilize? By default with it we will
> only interrupt when we see an empty ring.
> Which is liklely too much: pls try other values
> in the middle: e.g. make bufs half the ring,
> or bytes some small value like half ring * 200, or packets some
> small value etc.
>
> Set any one parameter to 0 to get current
> behaviour (interrupt immediately when enabled).
>
> Warning: completely untested.
>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>
> ---
>
> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> index aac05bc..6769cdc 100644
> --- a/drivers/vhost/net.c
> +++ b/drivers/vhost/net.c
> @@ -32,6 +32,13 @@
> * Using this limit prevents one virtqueue from starving others. */
> #define VHOST_NET_WEIGHT 0x80000
>
> +int tx_bytes_coalesce = 1000000000;
> +module_param(tx_bytes_coalesce, int, 0644);
> +int tx_bufs_coalesce = 1000000000;
> +module_param(tx_bufs_coalesce, int, 0644);
> +int tx_packets_coalesce = 1000000000;
> +module_param(tx_packets_coalesce, int, 0644);
> +
> enum {
> VHOST_NET_VQ_RX = 0,
> VHOST_NET_VQ_TX = 1,
> @@ -127,6 +134,9 @@ static void handle_tx(struct vhost_net *net)
> int err, wmem;
> size_t hdr_size;
> struct socket *sock;
> + int bytes_coalesced = 0;
> + int bufs_coalesced = 0;
> + int packets_coalesced = 0;
>
> /* TODO: check that we are running from vhost_worker? */
> sock = rcu_dereference_check(vq->private_data, 1);
> @@ -196,14 +206,26 @@ static void handle_tx(struct vhost_net *net)
> if (err != len)
> pr_debug("Truncated TX packet: "
> " len %d != %zd\n", err, len);
> - vhost_add_used_and_signal(&net->dev, vq, head, 0);
> total_len += len;
> + packets_coalesced += 1;
> + bytes_coalesced += len;
> + bufs_coalesced += out;
> + if (unlikely(packets_coalesced > tx_packets_coalesce ||
> + bytes_coalesced > tx_bytes_coalesce ||
> + bufs_coalesced > tx_bufs_coalesce))
> + vhost_add_used_and_signal(&net->dev, vq, head, 0);
I think the counters that exceed the limits need to be reset to 0 here.
Otherwise we keep signaling for every buffer once we hit this condition.
Thanks
Sridhar
> + else
> + vhost_add_used(vq, head, 0);
> if (unlikely(total_len >= VHOST_NET_WEIGHT)) {
> vhost_poll_queue(&vq->poll);
> break;
> }
> }
>
> + if (likely(packets_coalesced &&
> + bytes_coalesced &&
> + bufs_coalesced))
> + vhost_signal(&net->dev, vq);
> mutex_unlock(&vq->mutex);
> }
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Patch[1/1]:Adding_Netlink_Support_to_ThermalFramework
From: Thomas Renninger @ 2011-02-01 23:07 UTC (permalink / raw)
To: R, Durgadoss
Cc: Zhang, Rui, Henrique de Moraes Holschuh, Len Brown,
linux-acpi@vger.kernel.org
In-Reply-To: <D6D887BA8C9DFF48B5233887EF046541094494D951@bgsmsx502.gar.corp.intel.com>
Hi,
some quick comments:
> +The framework includes a simple notification mechanism, in the form of a
> +netlink event. Netlink socket initialization is done during the _init_
> +of the framework. Drivers which intend to use the notification mechanism
> +must register with the thermal framework. Once registered, they will be
> +assigned an unique id. They can call thermal_netlink_event() to send
> +netlink events.
> +
> +This thermal_netlink_event function takes two arguments:
> + 1.Originator id
My idea was: pass the thermal_zone_device id, compare with:
include/linux/thermal.h:
struct thermal_zone_device {
int id;
I expect it's used to create the sysfs directory:
/sys/devices/virtual/thermal/thermal_zone0
-> zero in the end should be the id (I did not verify this).
> +
> +enum thermal_devices {
> + /* For multi-core CPUs */
> + THERMAL_CPU_CORE0 = 0,
is this a placeholder for 32 cores? That's not much and not
generic enough. CPU should be enough. There also should
be THERMAL_DEVICE_UNKNOWN.
> + THERMAL_GPU = 32,
> + THERMAL_BATTERY,
> + /* Add other devices, if any */
> +};
> +
> +struct thermal_genl_event {
> + enum thermal_devices device;
> + enum thermal_events event;
> +};
> +
> +/* attributes of thermal_genl_family */
> +enum {
> + THERMAL_GENL_ATTR_UNSPEC,
> + THERMAL_GENL_ATTR_EVENT,
> + __THERMAL_GENL_ATTR_MAX,
> +};
> +#define THERMAL_GENL_ATTR_MAX (__THERMAL_GENL_ATTR_MAX - 1)
> +
> +/* commands supported by the thermal_genl_family */
> +enum {
> + THERMAL_GENL_CMD_UNSPEC,
> + THERMAL_GENL_CMD_EVENT,
> + __THERMAL_GENL_CMD_MAX,
> +};
> +#define THERMAL_GENL_CMD_MAX (__THERMAL_GENL_CMD_MAX - 1)
> +
> +int thermal_netlink_event(enum thermal_devices, enum thermal_events);
not sure what is best, an int or even better request:
struct thermal_zone_device* and then pass the ->id (additionally?)
to userspace. I see that there might be drivers which cannot register as a
thermal driver (e.g. getting an emergency notification from BIOS and no temperature
or thermal trip points at all) and that there might be exceptions needed for the
future, they could pass NULL or a specific value so that userspace knows it
cannot read up further info from /sys/devices/virtual/thermal/thermal_zone-1.
Just some thoughts, I somehow like the idea of userspace being
able to look up the thermal device in sysfs which has thrown the event.
I try to find some time to read into the user definable trip points
(THERMAL_CT_AUX1,..) and might be able to give more feedback then.
About the userspace tool that should pick up these events,
is this integrated into an existing project, is there
code already you could point to?
Thanks,
Thomas
^ permalink raw reply
* Re: [Qemu-devel] [PATCH] fix linuxboot.bin and multiboot.bin to not hijack int19
From: Anthony Liguori @ 2011-02-01 23:02 UTC (permalink / raw)
To: Gleb Natapov; +Cc: qemu-devel
In-Reply-To: <20110131090247.GT14750@redhat.com>
On 01/31/2011 03:02 AM, Gleb Natapov wrote:
> Currently linuxboot.bin and multiboot.bin option roms override int19
> vector to intercept boot process. No sane option rom should do that.
> Provide bev entry instead that will be called by BIOS if option rom
> is selected for booting.
>
> Signed-off-by: Gleb Natapov<gleb@redhat.com>
> ---
>
> Note that this patch should be applied after qemu will upgrade to Seabios
> that supports boot order. Otherwise there will be change in behavior
> since option rom will not be selected for booting by default.
>
I don't think there's a SeaBIOS release yet that includes the boot order
support.
Regards,
Anthony Liguori
> diff --git a/pc-bios/optionrom/linuxboot.S b/pc-bios/optionrom/linuxboot.S
> index c109363..79b3ae8 100644
> --- a/pc-bios/optionrom/linuxboot.S
> +++ b/pc-bios/optionrom/linuxboot.S
> @@ -22,6 +22,8 @@
>
> #include "optionrom.h"
>
> +#define PRODUCT "Linux loader"
> +
> BOOT_ROM_START
>
> run_linuxboot:
> diff --git a/pc-bios/optionrom/multiboot.S b/pc-bios/optionrom/multiboot.S
> index 9131837..069c4e8 100644
> --- a/pc-bios/optionrom/multiboot.S
> +++ b/pc-bios/optionrom/multiboot.S
> @@ -20,6 +20,8 @@
>
> #include "optionrom.h"
>
> +#define PRODUCT "multiboot loader"
> +
> #define MULTIBOOT_MAGIC 0x2badb002
>
> #define GS_PROT_JUMP 0
> diff --git a/pc-bios/optionrom/optionrom.h b/pc-bios/optionrom/optionrom.h
> index fbdd48a..95dbdda 100644
> --- a/pc-bios/optionrom/optionrom.h
> +++ b/pc-bios/optionrom/optionrom.h
> @@ -93,31 +93,41 @@
> .global _start; \
> _start:; \
> .short 0xaa55; \
> - .byte (_end - _start) / 512;
> + .byte (_end - _start) / 512; \
> + lret; \
> + .org 0x18; \
> + .short 0; \
> + .short _pnph; \
> + _pnph: \
> + .ascii "$PnP"; \
> + .byte 0x01; \
> + .byte ( _pnph_len / 16 ); \
> + .short 0x0000; \
> + .byte 0x00; \
> + .byte 0x00; \
> + .long 0x00000000; \
> + .short _manufacturer; \
> + .short _product; \
> + .long 0x00000000; \
> + .short 0x0000; \
> + .short 0x0000; \
> + .short _bev; \
> + .short 0x0000; \
> + .short 0x0000; \
> + .equ _pnph_len, . - _pnph;
>
> #define BOOT_ROM_START \
> OPTION_ROM_START \
> - push %eax; \
> - push %ds; \
> - \
> - /* setup ds so we can access the IVT */ \
> - xor %ax, %ax; \
> - mov %ax, %ds; \
> - \
> - /* install our int 19 handler */ \
> - movw $int19_handler, (0x19*4); \
> - mov %cs, (0x19*4+2); \
> - \
> - pop %ds; \
> - pop %eax; \
> - lret; \
> - \
> - int19_handler:; \
> + _bev:; \
> /* DS = CS */ \
> movw %cs, %ax; \
> movw %ax, %ds;
>
> #define OPTION_ROM_END \
> + _manufacturer:; \
> + .asciz "QEMU"; \
> + _product:; \
> + .asciz PRODUCT; \
> .align 512, 0; \
> _end:
>
> --
> Gleb.
>
>
^ permalink raw reply
* Re: Simple test system not working
From: Samuel Just @ 2011-02-01 23:01 UTC (permalink / raw)
Cc: ceph-devel
In-Reply-To: <loom.20110201T072445-567@post.gmane.org>
From the logs, it looks like the osd and mds are failing to bind to
an address. Perhaps b3 is not a valid host name?
-Sam
On 01/31/2011 10:48 PM, Chris Dunlop wrote:
> G'day,
>
> I'm just starting with ceph and am having trouble getting a test ceph
> installation going on debian lenny with a self-compiled 2.6.37. Ceph
> itself is latest unstable (73e76723) compiled using dpkg-buildpackage.
> The mon, mds and osd are all on the same machine. The ceph base /data
> directory is ext2 mounted with user_xattr. The ceph filesystem is
> created using the attached ceph.conf and:
>
> /sbin/mkcephfs -v -c /etc/ceph/ceph.conf --allhosts --mkbtrfs -k
> /etc/ceph/keyring.bin
>
> Once ceph has been started, trying a mount on a (separate) client
> machine (2.6.37 kernel) hangs before eventually giving an error:
>
> # mount -t ceph b3:/ /mnt
> mount error 5 = Input/output error
>
> On the server side, it seems that the cmon isn't accepting connections
> from the mds or osd. E.g. repeated instances of this in the osd log,
> and similar "mark_down"s in the mds log:
>
> 2011-02-01 17:18:47.463897 4a254950 -- 0.0.0.0:6801/7854 mark_down
> 10.200.63.129:6789/0 -- 0xca8a00
> 2011-02-01 17:18:47.463924 4a254950 -- 0.0.0.0:6801/7854 --> mon0
> 10.200.63.129:6789/0 -- auth(proto 0 26 bytes) v1 -- ?+0 0xcd9800
>
> The mon log (amongst a whole bunch of other stuff) repeats:
>
> 2011-02-01 17:18:51.445785 42c73950 mon.0@0(leader).mds e1 e1: 0/0/1 up
> 2011-02-01 17:18:51.445790 42c73950 mon.0@0(leader).osd e1 e1: 0 osds: 0 up, 0 in
>
> The attached logs.zip has the full logs from one attempted start.
>
> Is there something obvious I'm doing wrong?
>
> Thanks for any assistance!
>
> Chris
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [B.A.T.M.A.N.] [PATCH] batctl: add radiotap wifi packet decapsulation support
From: Sven Eckelmann @ 2011-02-01 23:02 UTC (permalink / raw)
To: b.a.t.m.a.n
Signed-off-by: Sven Eckelmann <sven@narfation.org>
---
batctl/tcpdump.c | 37 ++++++++++++++++++++++++++++++-------
batctl/tcpdump.h | 12 ++++++++++++
2 files changed, 42 insertions(+), 7 deletions(-)
diff --git a/batctl/tcpdump.c b/batctl/tcpdump.c
index 4443a2a..a715e27 100644
--- a/batctl/tcpdump.c
+++ b/batctl/tcpdump.c
@@ -460,6 +460,30 @@ static void parse_eth_hdr(unsigned char *packet_buff, ssize_t buff_len, int read
}
}
+static int monitor_header_length(unsigned char *packet_buff, ssize_t buff_len, int32_t hw_type)
+{
+ struct radiotap_header *radiotap_hdr;
+ switch (hw_type) {
+ case ARPHRD_IEEE80211_PRISM:
+ if (buff_len <= (ssize_t)PRISM_HEADER_LEN)
+ return -1;
+ else
+ return PRISM_HEADER_LEN;
+
+ case ARPHRD_IEEE80211_RADIOTAP:
+ if (buff_len <= (ssize_t)RADIOTAP_HEADER_LEN)
+ return -1;
+
+ radiotap_hdr = (struct radiotap_header*)packet_buff;
+ if (buff_len <= radiotap_hdr->it_len)
+ return -1;
+ else
+ return radiotap_hdr->it_len;
+ }
+
+ return -1;
+}
+
static void parse_wifi_hdr(unsigned char *packet_buff, ssize_t buff_len, int read_opt, int time_printed)
{
struct ether_header *eth_hdr;
@@ -468,12 +492,6 @@ static void parse_wifi_hdr(unsigned char *packet_buff, ssize_t buff_len, int rea
uint16_t fc;
int hdr_len;
- if (buff_len <= (ssize_t)PRISM_HEADER_LEN)
- return;
-
- buff_len -= PRISM_HEADER_LEN;
- packet_buff += PRISM_HEADER_LEN;
-
/* we assume a minimum size of 38 bytes
* (802.11 data frame + LLC)
* before we calculate the real size */
@@ -540,6 +558,7 @@ int tcpdump(int argc, char **argv)
int ret = EXIT_FAILURE, res, optchar, found_args = 1, max_sock = 0, tmp;
int read_opt = USE_BAT_HOSTS;
unsigned char packet_buff[2000];
+ int monitor_header_len = -1;
while ((optchar = getopt(argc, argv, "hnp:")) != -1) {
switch (optchar) {
@@ -609,6 +628,7 @@ int tcpdump(int argc, char **argv)
switch (dump_if->hw_type) {
case ARPHRD_ETHER:
case ARPHRD_IEEE80211_PRISM:
+ case ARPHRD_IEEE80211_RADIOTAP:
break;
default:
printf("Error - interface '%s' is of unknown type: %i\n", dump_if->dev, dump_if->hw_type);
@@ -685,7 +705,10 @@ int tcpdump(int argc, char **argv)
parse_eth_hdr(packet_buff, read_len, read_opt, 0);
break;
case ARPHRD_IEEE80211_PRISM:
- parse_wifi_hdr(packet_buff, read_len, read_opt, 0);
+ case ARPHRD_IEEE80211_RADIOTAP:
+ monitor_header_len = monitor_header_length(packet_buff, read_len, dump_if->hw_type);
+ if (monitor_header_len >= 0)
+ parse_wifi_hdr(packet_buff + monitor_header_len, read_len - monitor_header_len, read_opt, 0);
break;
default:
/* should not happen */
diff --git a/batctl/tcpdump.h b/batctl/tcpdump.h
index 17eaeb0..4364be1 100644
--- a/batctl/tcpdump.h
+++ b/batctl/tcpdump.h
@@ -27,6 +27,10 @@
#define ARPHRD_IEEE80211_PRISM 802
#endif
+#ifndef ARPHRD_IEEE80211_RADIOTAP
+#define ARPHRD_IEEE80211_RADIOTAP 803
+#endif
+
#define DUMP_TYPE_BATOGM 1
#define DUMP_TYPE_BATICMP 2
#define DUMP_TYPE_BATUCAST 4
@@ -67,6 +71,13 @@ struct ieee80211_hdr {
u_int8_t addr4[6];
} __attribute__ ((packed));
+struct radiotap_header {
+ u_int8_t it_version;
+ u_int8_t it_pad;
+ u_int16_t it_len;
+ u_int32_t it_present;
+} __attribute__((__packed__));
+
struct prism_item {
u_int32_t did;
u_int16_t status;
@@ -91,5 +102,6 @@ struct prism_header {
};
#define PRISM_HEADER_LEN sizeof(struct prism_header)
+#define RADIOTAP_HEADER_LEN sizeof(struct radiotap_header)
int tcpdump(int argc, char **argv);
--
1.7.2.3
^ permalink raw reply related
* Re: XFS internal error xfs_iformat(realtime) even after xfs_repair.
From: Dave Chinner @ 2011-02-01 23:04 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Ajeet Yadav, xfs
In-Reply-To: <20110201212832.GA1836@infradead.org>
On Tue, Feb 01, 2011 at 04:28:32PM -0500, Christoph Hellwig wrote:
> > It will be in the next release, whenever we make that.
> >
> > I've got to finish of the sync of the userspace/kernel code, there's
> > a few fixes already in the -dev branch as well, we need to pick up
> > the FIEMAP changes to xfs_io, and so on. We probably should get t???is
> > all done and plan for a new release in the next few weeks...
>
> I'm a bit concerned about the coverage of the libxfs resync. My
> preference would be to get out a minor release focusses at the repair
> fixes, then merge the resync to master directly after release and make
> sure it gets into all the test setups for a while before it gets
> released.
Sure, I have no problem with that - this is the first mention of a
new release and I was just mentioning everything that is happening
at the moment. Doing a point release with all the known
fixes before the large sync makes sense...
Cheers,
dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply
* Re: PCI passthrough issue
From: Jean Baptiste Favre @ 2011-02-01 23:01 UTC (permalink / raw)
To: xen-devel
In-Reply-To: <20110201193743.GA18656@dumpdata.com>
Le 01/02/2011 20:37, Konrad Rzeszutek Wilk a écrit :
> On Tue, Feb 01, 2011 at 04:23:09PM +0000, Ian Campbell wrote:
>> On Tue, 2011-02-01 at 15:38 +0000, Jean Baptiste Favre wrote:
>>> OK, just found it:
>>> after domU boot:
>>> - log in
>>> - ping -s 86 10.0.0.1 (fails)
>>> - rmmod sky2
>>> - modprobe sky2 copybreak=0 (no packet copied)
>>> - ping -s 86 10.0.0.1 (works)
>>>
>>> So it's clearly related to that option.
>>
>> Awesome!
>>
>>> Now the question: what am I supposed to do ?
>>
>> I think the next step is to try and reproduce on native 32 bit, with RAM
>> artificially limited via the mem= kernel command in option, this will
>> let us determine if this is a generic issue or is somehow Xen specific.
>>
>> The main difference caused by the copybreak option is that for larger
>> frames (i.e. always when copybreak==0) it hits a code path which uses
>> pci_map_single and pci_map_page to access the received data. When len <
>> copybreak it takes a path which uses pci_dma_sync_single_for_cpu, so it
>> seems like the later path is broken somehow. If the issue does turn out
>
> It could also have gotten the direction reverted (the 3c5XX code had it
> wrong at some point so..). Might make sense to compile the kernel with
> CONFIG_DMA_API_DEBUG which is good at detecting these issues.
>
>> to be related to Xen then I think that points to the swiotlb code.
>
>>
>> I assume you are not seeing "rx mapping error" in your domU dmesg? Did
>> you post a full guest console log at some point? Comparing the logs for
>> the 256MB, 398MB and 512MB guest RAM case might be useful.
>>
>> Konrad, is there some way to force swiotlb use even for native to make
>> the native test cases more relevant? Or is there some other direction we
>
> swiotlb=force will do it.
OK, just performed native test.
Installed 32bits Debian Squeeze, add 2.6.37 32bits kernel from
experimental, setup grub with following options:
"mem=256M swiotlb=force"
Ping tests work whatever can be packet size.
If I understood well what you explain to me, it's now clear that the
problem is somewhat Xen related, isn't it ?
I'll be able to continue any tests you want tomorrow.
Do you still need me to compile domU kernel with "CONFIG_DMA_API_DEBUG"
enabled ?
If so, what tools will I need to get debug informations ?
Now, it's time to sleep here in France :) (or I'll get killed by my girl
friend :D )
Regards,
JB
^ permalink raw reply
* Re: IPv6 filtering
From: Guido Winkelmann @ 2011-02-01 23:00 UTC (permalink / raw)
To: Jonathan Tripathy; +Cc: netfilter
In-Reply-To: <4D487D69.7000808@abpni.co.uk>
On Tuesday 01 February 2011 22:38:49 you wrote:
> With IPv6, does anyone have any experience on how to do this? I know we
> can use ip6tables, but isn't there some trickery with NDP (Which
> replaces ARP)?
There shouldn't be, just as long as you only filter on the source address of
outgoing packets, and not on the destination of incoming ones. The NDP request
packets go to weird multicast addresses that depend on the address being
requested. I haven't totally figured out the scheme yet (haven't really
tried).
Answer packets for NDP always have a valid IPv6 address from the answering
host as their source address. Also, they're IPv6 packets like any other and
not a separate protocol as with ARP+IPv4.
Note that hosts using IPv6 will usually have at least two autoconfigured
addresses, and it's sometimes hard to predict which one will be used as source
address for outgoing packets, especially if the number of configured addresses
grows. You must not block any of those.
Also see my message from yesterday on that subject, please. I'm having
problems with filtering IPv6 from VPSes as well, and if you find a solution
that works with large numbers of vpses, I would appreciate it if you could
share it.
Guido
^ permalink raw reply
* Re: 2.6.38-rc3: nouveau nv50 screen freeze
From: Ben Skeggs @ 2011-02-01 23:00 UTC (permalink / raw)
To: Alexey Dobriyan; +Cc: Linus Torvalds, linux-kernel
In-Reply-To: <20110201224147.GA4739@p183.telecom.by>
On Wed, 2011-02-02 at 00:41 +0200, Alexey Dobriyan wrote:
> nv50 (GeForce 8800 GTX) is broken.
> No switching to high-resolution fbcon occurs, screen freezes
> with 80x25 mode right before the switch (PCI bridge info in my case).
>
> Otherwise, box boots normally, allowing blind reboot.
This is already fixed in nouveau master, I'll get the patches to Linus
soon.
Thanks!
Ben.
>
> 6d86951a45013ac5b060c5e6307b11b7c685c76f is the first bad commit
> commit 6d86951a45013ac5b060c5e6307b11b7c685c76f
> Author: Ben Skeggs <bskeggs@redhat.com>
> Date: Wed Dec 8 11:19:30 2010 +1000
>
> drm/nvc0: initial support for tiled buffer objects
>
> Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
>
> diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.c b/drivers/gpu/drm/nouveau/nouveau_bo.c
> index 5dc639e..74d0ef4 100644
> --- a/drivers/gpu/drm/nouveau/nouveau_bo.c
> +++ b/drivers/gpu/drm/nouveau/nouveau_bo.c
> @@ -120,9 +120,6 @@ nouveau_bo_new(struct drm_device *dev, struct nouveau_channel *chan,
> align >>= PAGE_SHIFT;
>
> if (!nvbo->no_vm && dev_priv->chan_vm) {
> - if (dev_priv->card_type == NV_C0)
> - page_shift = 12;
> -
> ret = nouveau_vm_get(dev_priv->chan_vm, size, page_shift,
> NV_MEM_ACCESS_RW, &nvbo->vma);
> if (ret) {
> diff --git a/drivers/gpu/drm/nouveau/nouveau_dma.h b/drivers/gpu/drm/nouveau/nouveau_dma.h
> index c118a33..c36f176 100644
> --- a/drivers/gpu/drm/nouveau/nouveau_dma.h
> +++ b/drivers/gpu/drm/nouveau/nouveau_dma.h
> @@ -77,7 +77,8 @@ enum {
> /* G80+ display objects */
> NvEvoVRAM = 0x01000000,
> NvEvoFB16 = 0x01000001,
> - NvEvoFB32 = 0x01000002
> + NvEvoFB32 = 0x01000002,
> + NvEvoVRAM_LP = 0x01000003
> };
>
> #define NV_MEMORY_TO_MEMORY_FORMAT 0x00000039
> diff --git a/drivers/gpu/drm/nouveau/nv50_crtc.c b/drivers/gpu/drm/nouveau/nv50_crtc.c
> index 2c346f7..9023c4d 100644
> --- a/drivers/gpu/drm/nouveau/nv50_crtc.c
> +++ b/drivers/gpu/drm/nouveau/nv50_crtc.c
> @@ -115,15 +115,16 @@ nv50_crtc_blank(struct nouveau_crtc *nv_crtc, bool blanked)
> OUT_RING(evo, 0);
> BEGIN_RING(evo, 0, NV50_EVO_CRTC(index, FB_DMA), 1);
> if (dev_priv->chipset != 0x50)
> - if (nv_crtc->fb.tile_flags == 0x7a00)
> + if (nv_crtc->fb.tile_flags == 0x7a00 ||
> + nv_crtc->fb.tile_flags == 0xfe00)
> OUT_RING(evo, NvEvoFB32);
> else
> if (nv_crtc->fb.tile_flags == 0x7000)
> OUT_RING(evo, NvEvoFB16);
> else
> - OUT_RING(evo, NvEvoVRAM);
> + OUT_RING(evo, NvEvoVRAM_LP);
> else
> - OUT_RING(evo, NvEvoVRAM);
> + OUT_RING(evo, NvEvoVRAM_LP);
> }
>
> nv_crtc->fb.blanked = blanked;
> @@ -555,13 +556,14 @@ nv50_crtc_do_mode_set_base(struct drm_crtc *crtc,
> return ret;
>
> BEGIN_RING(evo, 0, NV50_EVO_CRTC(nv_crtc->index, FB_DMA), 1);
> - if (nv_crtc->fb.tile_flags == 0x7a00)
> + if (nv_crtc->fb.tile_flags == 0x7a00 ||
> + nv_crtc->fb.tile_flags == 0xfe00)
> OUT_RING(evo, NvEvoFB32);
> else
> if (nv_crtc->fb.tile_flags == 0x7000)
> OUT_RING(evo, NvEvoFB16);
> else
> - OUT_RING(evo, NvEvoVRAM);
> + OUT_RING(evo, NvEvoVRAM_LP);
> }
>
> ret = RING_SPACE(evo, 12);
> @@ -575,8 +577,10 @@ nv50_crtc_do_mode_set_base(struct drm_crtc *crtc,
> if (!nv_crtc->fb.tile_flags) {
> OUT_RING(evo, drm_fb->pitch | (1 << 20));
> } else {
> - OUT_RING(evo, ((drm_fb->pitch / 4) << 4) |
> - fb->nvbo->tile_mode);
> + u32 tile_mode = fb->nvbo->tile_mode;
> + if (dev_priv->card_type >= NV_C0)
> + tile_mode >>= 4;
> + OUT_RING(evo, ((drm_fb->pitch / 4) << 4) | tile_mode);
> }
> if (dev_priv->chipset == 0x50)
> OUT_RING(evo, (nv_crtc->fb.tile_flags << 8) | format);
> diff --git a/drivers/gpu/drm/nouveau/nv50_evo.c b/drivers/gpu/drm/nouveau/nv50_evo.c
> index 887b2a9..14e24e9 100644
> --- a/drivers/gpu/drm/nouveau/nv50_evo.c
> +++ b/drivers/gpu/drm/nouveau/nv50_evo.c
> @@ -53,7 +53,8 @@ nv50_evo_channel_del(struct nouveau_channel **pevo)
>
> int
> nv50_evo_dmaobj_new(struct nouveau_channel *evo, u32 class, u32 name,
> - u32 tile_flags, u32 magic_flags, u32 offset, u32 limit)
> + u32 tile_flags, u32 magic_flags, u32 offset, u32 limit,
> + u32 flags5)
> {
> struct drm_nouveau_private *dev_priv = evo->dev->dev_private;
> struct drm_device *dev = evo->dev;
> @@ -70,10 +71,7 @@ nv50_evo_dmaobj_new(struct nouveau_channel *evo, u32 class, u32 name,
> nv_wo32(obj, 8, offset);
> nv_wo32(obj, 12, 0x00000000);
> nv_wo32(obj, 16, 0x00000000);
> - if (dev_priv->card_type < NV_C0)
> - nv_wo32(obj, 20, 0x00010000);
> - else
> - nv_wo32(obj, 20, 0x00020000);
> + nv_wo32(obj, 20, flags5);
> dev_priv->engine.instmem.flush(dev);
>
> ret = nouveau_ramht_insert(evo, name, obj);
> @@ -264,9 +262,31 @@ nv50_evo_create(struct drm_device *dev)
> }
>
> /* create some default objects for the scanout memtypes we support */
> + if (dev_priv->card_type >= NV_C0) {
> + ret = nv50_evo_dmaobj_new(evo, 0x3d, NvEvoFB32, 0xfe, 0x19,
> + 0, 0xffffffff, 0x00000000);
> + if (ret) {
> + nv50_evo_channel_del(&dev_priv->evo);
> + return ret;
> + }
> +
> + ret = nv50_evo_dmaobj_new(evo, 0x3d, NvEvoVRAM, 0, 0x19,
> + 0, dev_priv->vram_size, 0x00020000);
> + if (ret) {
> + nv50_evo_channel_del(&dev_priv->evo);
> + return ret;
> + }
> +
> + ret = nv50_evo_dmaobj_new(evo, 0x3d, NvEvoVRAM_LP, 0, 0x19,
> + 0, dev_priv->vram_size, 0x00000000);
> + if (ret) {
> + nv50_evo_channel_del(&dev_priv->evo);
> + return ret;
> + }
> + } else
> if (dev_priv->chipset != 0x50) {
> ret = nv50_evo_dmaobj_new(evo, 0x3d, NvEvoFB16, 0x70, 0x19,
> - 0, 0xffffffff);
> + 0, 0xffffffff, 0x00010000);
> if (ret) {
> nv50_evo_channel_del(&dev_priv->evo);
> return ret;
> @@ -274,18 +294,25 @@ nv50_evo_create(struct drm_device *dev)
>
>
> ret = nv50_evo_dmaobj_new(evo, 0x3d, NvEvoFB32, 0x7a, 0x19,
> - 0, 0xffffffff);
> + 0, 0xffffffff, 0x00010000);
> if (ret) {
> nv50_evo_channel_del(&dev_priv->evo);
> return ret;
> }
> - }
>
> - ret = nv50_evo_dmaobj_new(evo, 0x3d, NvEvoVRAM, 0, 0x19,
> - 0, dev_priv->vram_size);
> - if (ret) {
> - nv50_evo_channel_del(&dev_priv->evo);
> - return ret;
> + ret = nv50_evo_dmaobj_new(evo, 0x3d, NvEvoVRAM, 0, 0x19,
> + 0, dev_priv->vram_size, 0x00010000);
> + if (ret) {
> + nv50_evo_channel_del(&dev_priv->evo);
> + return ret;
> + }
> +
> + ret = nv50_evo_dmaobj_new(evo, 0x3d, NvEvoVRAM_LP, 0, 0x19,
> + 0, dev_priv->vram_size, 0x00010000);
> + if (ret) {
> + nv50_evo_channel_del(&dev_priv->evo);
> + return ret;
> + }
> }
>
> return 0;
> diff --git a/drivers/gpu/drm/nouveau/nvc0_vram.c b/drivers/gpu/drm/nouveau/nvc0_vram.c
> index 41fcae5..858eda5 100644
> --- a/drivers/gpu/drm/nouveau/nvc0_vram.c
> +++ b/drivers/gpu/drm/nouveau/nvc0_vram.c
> @@ -29,8 +29,16 @@
> bool
> nvc0_vram_flags_valid(struct drm_device *dev, u32 tile_flags)
> {
> - if (likely(!(tile_flags & NOUVEAU_GEM_TILE_LAYOUT_MASK)))
> + switch (tile_flags & NOUVEAU_GEM_TILE_LAYOUT_MASK) {
> + case 0x0000:
> + case 0xfe00:
> + case 0xdb00:
> + case 0x1100:
> return true;
> + default:
> + break;
> + }
> +
> return false;
> }
>
^ permalink raw reply
* Re: OE TSC
From: Richard Purdie @ 2011-02-01 22:59 UTC (permalink / raw)
To: openembedded-devel
In-Reply-To: <4D487BA0.5080101@opensimpad.org>
On Tue, 2011-02-01 at 22:31 +0100, Bernhard Guillon wrote:
> On 01.02.2011 21:08, Frans Meulenbroeks wrote:
> > ... a team that
> > is chosen by the community is much more empowered to make decisions.
> >
> +1
>
> In my opinion it is very important in the current situation to have a vote.
Just please keep in mind what this looks like to the outside world.
So far various amounts of bad timing, unfortunate circumstances and the
rules have meant that 2-4 months on we could appear to be going around
in circles and unlikely to make a "real" decision soon (at least six
weeks for a vote).
Yocto talks to people commercially and 3-5 months to make some decisions
(not implement, just make the decisions) would be commercial suicide to
a business.
I accept this takes as long as it takes and I'm not going to even
attempt to rush anything but OE isn't doing its perception much good to
be brutally honest.
On the plus side I know there are positive moves afoot too so its not
all doom and gloom but its worth keeping the above in mind.
Cheers,
Richard
^ permalink raw reply
* Re: [PATCH 6/6] xfs: remove handling of duplicates the busy extent tree
From: Alex Elder @ 2011-02-01 23:02 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: xfs
In-Reply-To: <20110121092551.629220945@bombadil.infradead.org>
On Fri, 2011-01-21 at 04:22 -0500, Christoph Hellwig wrote:
> plain text document attachment (xfs-remove-duplicate-extent-handling)
> With the recent changes we never re-used busy extents. Remove all handling
> of them and replace them with asserts. This also effectively reverts
> commit 955833cf2ad0aa39b336e853cad212d867199984
>
> "xfs: make the log ticket ID available outside the log infrastructure"
I have a hunch that this, for you, was an itch that
needed scratching.
Anyway, I was wondering about the following when
xfs_alloc_busy_search_trim() was added. Maybe you
could just offer a quick explanation. If we always
skip busy extents when allocating blocks, will we
still satisfy allocation requests when we're almost
out of space? Will a log flush cause busy extents
to become non-busy when we find nothing available,
thereby always satifying requests we would have
satisfied (perhaps with busy extents) previously?
Second, is there *anything* that could be said about
the resulting allocation patterns? Will they be
affected in any adverse (or beneficial) way by
skipping busy extents? (Will we get more fragmented
earlier, for a contrived example?) I have no
feeling for it but thought the question ought
to be asked...
Outside of the above, I think this patch looks fine.
The xfs_alloc_busy_search() call becomes a debug-only
function (and might as well be surrounded by DEBUG
ifdef's, accordingly), ensuring an allocated extent
is never found in the busy list.
I'm not signing off on this for now, though, because
I want to hear back on the comments from the previous
patch, and I think you're going to be re-posting the
series anyway.
-Alex
> Signed-off-by: Christoph Hellwig <hch@lst.de>
>
> Index: xfs/fs/xfs/xfs_ag.h
> ==========================================
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply
* Re: Network performance with small packets
From: Shirley Ma @ 2011-02-01 22:59 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Sridhar Samudrala, Steve Dobbelstein, David Miller, kvm, mashirle,
netdev
In-Reply-To: <20110201215603.GA31348@redhat.com>
On Tue, 2011-02-01 at 23:56 +0200, Michael S. Tsirkin wrote:
> There are flags for bytes, buffers and packets.
> Try playing with any one of them :)
> Just be sure to use v2.
>
>
> >I would like to change it to
> > half of the ring size instead for signaling. Is that OK?
> >
> > Shirley
> >
> >
>
> Sure that is why I made it a parameter so you can experiment.
The initial test results shows that the CPUs utilization has been
reduced some, and BW has increased some with the default parameters,
like 1K message size BW goes from 2.5Gb/s about 2.8Gb/s, CPU utilization
down from 4x% to 38%, (Similar results from the patch I submitted a
while ago to reduce signaling on vhost) but far away from dropping
packet results.
I am going to change the code to use 1/2 ring size to wake the netif
queue.
Shirley
^ permalink raw reply
* Re: [PATCH 5/6] xfs: do not classify freed allocation btree blocks as busy
From: Alex Elder @ 2011-02-01 23:02 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: xfs
In-Reply-To: <20110121092551.402449845@bombadil.infradead.org>
On Fri, 2011-01-21 at 04:22 -0500, Christoph Hellwig wrote:
> During an allocation or freeing of an extent, we occasionally have an
> interesting situation happen during a btree update. We may merge a block
> as part of a record delete, then allocate it again immediately afterwards
> when inserting a modified extent. xfs_alloc_fixup_trees() does this sort
> of manipulation to the btrees, and so can merge then immediately split
> the tree resulting the in the same busy block being used from the
> freelist.
>
> Previously, this would trigger a synchronous log force of the entire log
> (as the current transaction had a lsn of 0 until committed), but continue
> to allow the extent to be used because if it is not on disk now then it
> must have been freed and marked busy in this transaction.
>
> In the case of allocbt blocks, we log the fact that they get moved to the
> freelist and we also log them when the move off the free list back into
> the free space trees. When we move the blocks off the freelist back to
> the trees, we mark the extent busy at that point so that it doesn't get
> reused until the transaction is committed.
>
> This means that as long as the block is on the AGFL and is only used
> for allocbt blocks, we don't have to force the log to ensure prior
> manipulating transactions are on disk as the process of log recovery
> will replay the transactions in the correct order.
>
> However, we still need to protect against the fact that as we approach
> ENOSPC in an AG we can allocate data and metadata blocks direct from the
> AGFL. In this case, we need to treat btree blocks just like any other
> busy extent. Hence we still need to track btree blocks in the busy extent
> tree, but we need to distinguish them from normal extents so we can apply
> the necessary exceptions for btree block allocation.
OK, it's taken a while but I'm finally ready to comment on
these last two patches. I've been thinking about how these
freed blocks ought to be handled, but that's really a separate
discussion so I'll stick to the patch at hand for now.
It has a few problems. I also think it may be making an
already a bit confusing situation possibly less obvious.
What I mean is, there are some rules we're depending on
here and those rules aren't very explicitly documented
very well in the code. (Like, "if a btree node block
is freed it goes into the freelist, and in that case it
needs to be marked busy; normally the log will need to
be forced if that block gets reallocated later, but not
if it is getting reallocated later for use in the
allocation btree.") It's not the nicest thing to
have to try to explain, but that means it's even
more important to do so.
I think some of the commit explanations help, but I
think the code itself ought to be more clear about
what's going on.
> Signed-off-by: Dave Chinner <david@fromorbit.com>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
>
> Index: xfs/fs/xfs/xfs_ag.h
> ===================================================================
> --- xfs.orig/fs/xfs/xfs_ag.h 2011-01-17 22:03:15.000000000 +0100
> +++ xfs/fs/xfs/xfs_ag.h 2011-01-17 22:32:06.541004201 +0100
> @@ -188,6 +188,11 @@ struct xfs_busy_extent {
> xfs_agblock_t bno;
> xfs_extlen_t length;
> xlog_tid_t tid; /* transaction that created this */
> + int flags;
> +};
> +
> +enum {
> + XFS_BUSY_FREELIST = 0x0001, /* busy extent on freelist from abt */
I don't really like the name here. It emphasizes the freelist
rather than the fact that the extent was freed from the
allocation btree, which is I think the more important bit.
I think it's named this because it represents who called
xfs_alloc_busy_insert() (xfs_alloc_put_freelist(), as
opposed to xfs_free_ag_extent() which would pass 0 for
its "flags" value). There's probably not a good concise
name to represent that call site, but I think I'd prefer
to have one rather than just passing 0.
In any case, the real distinction is whether the extent
being marked busy is a block previously used for the
allocation btree or not.
> };
>
> /*
> Index: xfs/fs/xfs/xfs_alloc.c
> ===================================================================
> --- xfs.orig/fs/xfs/xfs_alloc.c 2011-01-17 22:30:54.000000000 +0100
> +++ xfs/fs/xfs/xfs_alloc.c 2011-01-17 22:39:29.021005529 +0100
Note that there are some comments in this file that begin
with:
* AG Busy list management
They ought to be updated to give a one-line description of
the new functions.
. . .
> @@ -1996,22 +2000,23 @@ xfs_alloc_get_freelist(
> xfs_alloc_log_agf(tp, agbp, logflags);
> *bnop = bno;
>
> - /*
> - * As blocks are freed, they are added to the per-ag busy list and
> - * remain there until the freeing transaction is committed to disk.
> - * Now that we have allocated blocks, this list must be searched to see
> - * if a block is being reused. If one is, then the freeing transaction
> - * must be pushed to disk before this transaction.
> - *
> - * We do this by setting the current transaction to a sync transaction
> - * which guarantees that the freeing transaction is on disk before this
> - * transaction. This is done instead of a synchronous log force here so
> - * that we don't sit and wait with the AGF locked in the transaction
> - * during the log force.
> - */
> if (type != XFS_FREELIST_BALANCE) {
> - if (xfs_alloc_busy_search(mp, agno, bno, 1))
> - xfs_trans_set_sync(tp);
> + /*
> + * If this block is marked busy we may have to force out the
> + * log to prevent reuse until the freeing transaction has been
> + * commited.
> + *
> + * If we're just allocating the block to rebalance the the
> + * freelist size we can skip this exercise as the block
> + * just keeps its busy marking while migrating to the
> + * allocation btree.
> + *
> + * If the block was freed from a btree and gets reallocated
> + * to it we may skip the log force, but details are handled
> + * by xfs_alloc_busy_force.
> + */
> + xfs_alloc_busy_force(mp, agno, bno, 1,
> + type == XFS_FREELIST_BTREE);
With the "if (type...)" above and the conditional expression
being passed as the last argument here you're basically
enumerating all possible values of "type". I think it might
look cleaner to just pass type as-is, and let xfs_alloc_busy_force()
hide this part of the logic also.
> }
>
> return 0;
. . .
> @@ -2123,6 +2129,14 @@ xfs_alloc_put_freelist(
> (int)((xfs_caddr_t)blockp - (xfs_caddr_t)agfl),
> (int)((xfs_caddr_t)blockp - (xfs_caddr_t)agfl +
> sizeof(xfs_agblock_t) - 1));
> +
> + /*
> + * If it's a btree block, mark it busy as it has just been freed.
> + * Otherwise we are just replenishing the free list with extents
> + * that are already free so we don't need to track them.
> + */
> + if (btreeblk)
> + xfs_alloc_busy_insert(tp, agno, bno, 1, XFS_BUSY_FREELIST);
On second thought... You could just pass the value of btreeblk
as the last argument here and let xfs_alloc_busy_insert() decide
whether inserting it is needed or not.
> return 0;
> }
>
. . .
> @@ -2733,6 +2749,62 @@ xfs_alloc_busy_search(
> return match;
> }
>
> +STATIC void
> +xfs_alloc_busy_force(
It's a shame this and xfs_alloc_busy_search() (as well
as the *_trim() variant) can't rely on a common helper
function, since they rely on basically the same logic
for searching the tree.
> + struct xfs_mount *mp,
> + xfs_agnumber_t agno,
> + xfs_agblock_t bno,
> + xfs_extlen_t len,
> + int btreeblk)
> +{
> + struct xfs_perag *pag;
> + struct rb_node *rbp;
> + struct xfs_busy_extent *busyp;
> + int match = 0;
> +
> + pag = xfs_perag_get(mp, agno);
> + spin_lock(&pag->pagb_lock);
> +
> + rbp = pag->pagb_tree.rb_node;
> +
> + /* find closest start bno overlap */
> + while (rbp) {
> + busyp = rb_entry(rbp, struct xfs_busy_extent, rb_node);
> + if (bno < busyp->bno) {
> + /* may overlap, but exact start block is lower */
> + if (bno + len > busyp->bno) {
> + match = 1;
> + break;
> + }
> + rbp = rbp->rb_left;
> + } else if (bno > busyp->bno) {
> + /* may overlap, but exact start block is higher */
> + if (bno < busyp->bno + busyp->length) {
> + match = 1;
> + break;
> + }
> + rbp = rbp->rb_right;
> + } else {
> + /* bno matches busyp, length determines exact match */
> + match = 1;
I believe this is not sufficient. You need to check for
an exact match. More below.
> + break;
> + }
> + }
> +
> + if (match && btreeblk && (busyp->flags & XFS_BUSY_FREELIST)) {
> + list_del_init(&busyp->list);
If xfs_alloc_busy_insert() finds that an extent being marked
busy abuts or overlaps an existing busy extent entry, it
combines the two. Because of this, I think it's possible that
a busy entry found to overlap the range provided here (i.e.,
[bno, bno+len)) may contain more than just the overlapping
region. Specifically this case will be freeing a single block,
and what I'm saying is that this single block could have been
merged with another by a different call to xfs_alloc_busy_insert().
So simply deleting the entire entry is wrong here. You have
to see if we're in the situation I describe, and handle it
accordingly. That gets messy, unfortunately.
This also highlights another issue with the addition of
the "flags" value to the xfs_busy_extent structure. It
is no longer valid to simply combine two busy entries.
You only can combine them if their flag values match.
Otherwise, when combining them you could either gain
or lose flag values associated with the involved extents.
> + rb_erase(&busyp->rb_node, &pag->pagb_tree);
> + kmem_free(busyp);
> + match = 0;
> + }
> + spin_unlock(&pag->pagb_lock);
> + trace_xfs_alloc_busysearch(mp, agno, bno, len, !!match);
> + xfs_perag_put(pag);
> +
> + if (match)
> + xfs_log_force(mp, XFS_LOG_SYNC);
I'm not sure how much difference it makes, but why can't
we just call xfs_log_force_lsn(mp, lsn, flags), where
"lsn" is the sequence number for the operation that
freed the extent that's being re-allocated. (This
applies to the existing code as well, not just this
change.) This suggestion also gets tricky due to
merging busy extents (similar to what I said about
"flags" above, but in this case it's the lsn).
> +}
> +
> /*
> * For a given extent [fbno, flen], search the busy extent list
> * to find a subset of the extent that is no
. . .
-Alex
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply
* [PATCH] Support different branch layouts in git-p4
From: Ian Wienand @ 2011-02-01 22:59 UTC (permalink / raw)
To: git
Hi,
I think the addition to the git-p4.txt in the diff explains the
reasoning behind the patch best. In short, we have a repository
layout
//depot/foo/branch
//depot/moo/branch
where we require projects 'foo' and 'moo' to be alongside each other.
We can do this with p4 views, but currently have to have 'foo' and
'moo' in separate git repos.
This just munges the incoming paths to either put the branch as the
top level directory, or just remove it entirely if you don't need it.
I've tested it locally, but I don't really have a wide variety of p4
environments to expose it too.
-i
Signed-off-by: Ian Wienand <ianw@vmware.com>
---
contrib/fast-import/git-p4 | 35 +++++++++++++++++++++++-
contrib/fast-import/git-p4.txt | 58 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 92 insertions(+), 1 deletions(-)
diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index 04ce7e3..4bd40f8 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -848,6 +848,10 @@ class P4Sync(Command):
optparse.make_option("--max-changes", dest="maxChanges"),
optparse.make_option("--keep-path", dest="keepRepoPath", action='store_true',
help="Keep entire BRANCH/DIR/SUBDIR prefix during import"),
+ optparse.make_option("--branch-path", dest="branchPath", type='choice',
+ choices=('none', 'first'),
+ default=None,
+ help="Remove the branch dir (none) or move it above project dir (first)"),
optparse.make_option("--use-client-spec", dest="useClientSpec", action='store_true',
help="Only sync files that are included in the Perforce Client Spec")
]
@@ -917,6 +921,20 @@ class P4Sync(Command):
if path.startswith(p):
path = path[len(p):]
+ # reorg to move/remove branch from the output filename -- kind
+ # of like how you can set your view in your p4 client
+ if self.keepRepoPath and self.branchPath == 'first':
+ # move the second element first, so what was was
+ # "//depot/proj/branch/file" becomes "branch/proj/file".
+ path = re.sub("^([^/]+/)([^/]+/)", r'\2\1', path)
+ elif self.keepRepoPath and self.branchPath == 'none':
+ # remove the second element, so what was
+ # "//depot/proj/branch/file" becomes "proj/file"
+ path = re.sub("^([^/]+/)([^/]+/)", r'\2', path)
+ elif self.branchPath:
+ sys.stderr.write("branchPath without keepRepoPath?")
+ sys.exit(1)
+
return path
def splitFilesIntoBranches(self, commit):
@@ -940,7 +958,6 @@ class P4Sync(Command):
relPath = self.stripRepoPath(path, self.depotPaths)
for branch in self.knownBranches.keys():
-
# add a trailing slash so that a commit into qt/4.2foo doesn't end up in qt/4.2
if relPath.startswith(branch + "/"):
if branch not in branches:
@@ -1283,12 +1300,24 @@ class P4Sync(Command):
if self.keepRepoPath:
option_keys['keepRepoPath'] = 1
+ # since we're just saving the dict keys, append the branchPath
+ # option to the key
+ if self.branchPath:
+ option_keys['branchPath_%s' % self.branchPath] = 1
+
d["options"] = ' '.join(sorted(option_keys.keys()))
def readOptions(self, d):
self.keepRepoPath = (d.has_key('options')
and ('keepRepoPath' in d['options']))
+ # restore the branchpath option; is one of "none" and "first"
+ if (d.has_key('options')):
+ if ('branchPath_none' in d['options']):
+ self.branchPath = 'none'
+ elif ('branchPath_first' in d['options']):
+ self.branchPath = 'first'
+
def gitRefForBranch(self, branch):
if branch == "main":
return self.refPrefix + "master"
@@ -1775,6 +1804,10 @@ class P4Clone(P4Sync):
sys.stderr.write("Must specify destination for --keep-path\n")
sys.exit(1)
+ if self.branchPath and not self.keepRepoPath:
+ sys.stderr.write("Must specify --keep-path for --branch-path\n")
+ sys.exit(1)
+
depotPaths = args
if not self.cloneDestination and len(depotPaths) > 1:
diff --git a/contrib/fast-import/git-p4.txt b/contrib/fast-import/git-p4.txt
index 49b3359..669c63c 100644
--- a/contrib/fast-import/git-p4.txt
+++ b/contrib/fast-import/git-p4.txt
@@ -191,6 +191,64 @@ git-p4.useclientspec
git config [--global] git-p4.useclientspec false
+Dealing with different repository layouts
+=========================================
+
+Perforce clients can map views of projects and branches in different
+ways which your build system may rely on. Say your code is organised
+as two projects "foo" and "moo" which have a common branch
+
+//depot/foo/branch/...
+//depot/moo/branch/...
+
+and you require both "foo" and "moo" projects in your git repository,
+there are several options.
+
+Firstly, you could simply clone each project as a completely separate
+git tree. However, if the two projects are dependent on each other
+this can be annoying for both sync -- you must remember to sync both
+"foo" and "moo" to keep everything consistent -- and submit -- a
+change that should logically be a single changeset across "foo" and
+"moo" will have to be broken up (breaking bisection too).
+
+Another option is to simply specify multiple depots
+
+ git p4 sync //depot/foo/branch //depot/moo/branch
+
+which will import "foo" and "moo" into the same directory.
+
+To keep the projects separate, the --keep-path option used as
+
+ git p4 sync --keep-path --destination /tmp/boo/ //depot/foo/branch //depot/moo/branch
+
+will create a layout of
+
+ /tmp/boo/foo/branch/...
+ /tmp/boo/moo/branch/...
+
+However, some build systems may rely on p4's ability to specify
+destinations for views in your client. The --branch-path flag, which
+requires the --keep-path flag, allows two additional layout options.
+
+ git p4 sync --keep-path --destination /tmp/boo --branch-path=none //depot/foo/branch //depot/moo/branch
+
+will remove the branch name entirely, leaving you with a directory
+that looks like
+
+ /tmp/boo/foo/...
+ /tmp/boo/moo/...
+
+and
+
+ git p4 sync --keep-path --destination /tmp/boo --branch-path=first //depot/foo/branch //depot/moo/branch
+
+will give you each of the projects under a directory named for their
+common branch
+
+ /tmp/boo/branch/foo/...
+ /tmp/boo/branch/moo/...
+
+
Implementation Details...
=========================
--
1.7.2.3
^ permalink raw reply related
* big files in git was: Re: Features from GitSurvey 2010
From: david @ 2011-02-01 22:50 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Jonathan Nieder, Dmitry S. Kravtsov, git
In-Reply-To: <201102011451.17456.jnareb@gmail.com>
On Tue, 1 Feb 2011, Jakub Narebski wrote:
> On Sun, 30 Jan 2011, Jonathan Nieder wrote:
>> Hi Dmitry,
>>
>> Dmitry S. Kravtsov wrote:
>>
>>> I want to dedicate my coursework at University to implementation of
>>> some useful git feature. So I'm interesting in some kind of list of
>>> development status of these features
>> [...]
>>> Or I'll be glad to know what features are now 'free' and what are
>>> currently in active development.
>>
>> Interesting question. The short answer is that they are all "free".
>> Generally people seem to be happy to learn of an alternative approach
>> to what they have been working on.
>>
>> [For the following pointers, the easiest way to follow up is probably
>> to search the mailing list archives.]
>>
>>> better support for big files (large media)
>>
>> For a conservative approach, you might want to get in touch with Sam
>> Hocevar, Nicolas Pitre, and Miklos Vajna. The idea is to stream big
>> files directly to pack and not waste time trying to compress them.
>
> There is also, supposedly stalled, git-bigfiles project.
why is the clean/smudge approach that came through the list a week or two
ago not acceptable?
While people talked about how it would be nice to store the large files on
$remote_destination, just create a .git/bigfiles and store them in there.
with the ability to pass the filename to the clean/smudge scripts, you can
even avoid the copy (replacing it with a mv) and have a working, if
bare-bones system.
Then people can create/submit enhanced versions of these scripts that
store the large files elsewhere if they want, but we would be past the
"git can't handle large files" into "git handles large files less
efficiently", which is a much better place to be.
If nobody else has time to take those e-mails and create a set of
clean/smudge scripts, I'll do so later this week (unless there is some
reason why they wouldn't be acceptable)
I guess the only question is how to tell what files need to be handled
this way, but can't we have something in .gitattributes about the file
size? (and if that's a problem for checking files out, have the stored
file be a sparse file, that way it's large, but doesn't take much space on
sane filesystems)
David Lang
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.