public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
* [RFC PATCH 00/12] xfs_db: write support for v5 filesystems
@ 2013-09-05 12:13 Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 01/12] db: separate out straight buffer IO from map based IO Dave Chinner
                   ` (11 more replies)
  0 siblings, 12 replies; 13+ messages in thread
From: Dave Chinner @ 2013-09-05 12:13 UTC (permalink / raw)
  To: xfs

Hi folks,

This RFC series brings write support to xfs_db for CRC enabled
filesytsems. The series converts xfs_db to use libxfs for it's IO,
adds awareness of verifiers and CRC validation to the IO stack and
enables write support for v5 filesystem structures.

The main infrastructure change is switching to libxfs-based IO
instead of buffered block device IO. This means that xfs_db now
switches to direct IO cached in it's own address space, meaning that
when it runs it always starts with a cold cache and so we don't have
problem with it reading stale cached data out of the kernel.

With it using libxfs, we can also attach verifiers to the IO to
validate structures and CRCs before we act on them. The fact that
CRCs were passed or not is available to usres of the data that is
read, and so programs like xfs_check can be made to fail if CRC
validation fails on read of metadata.

Finally, write support is enabled because now that all data buffers
have verifiers they can recalculate CRCs on write. There's some
special support for inodes and dquots, but they will still have CRCs
recalculated before write IOs. Hence xfs_db won't corrupt v5
metadata now when you change a field value manually.

There are still some issues with the libxfs support - there are some
places where xfs_db re-reads the same block address with different
buffer lengths, resulting in libxfs throwing a cache lookup badness
warning. e.g:

$ sudo xfs_db -c "push sb" -c "p crc" /dev/vdc
7fe16ab3f740: Badness in key lookup (length)
bp=(bno 0x0, len 0 bytes) key=(bno 0x0, len 512 bytes)
Version 5 superblock detected. xfsprogs has EXPERIMENTAL support enabled!
Use of these features is at your own risk!
crc = 0xfc7b42ec (correct)
$

This shows a buffer at block 0 was created with zero length for some
reason, but when we push the superblock onto the stack, it looks up
a buffer of 512 bytes, not 0, and it throws an error. The
experimental warning is being thrown by the superblock buffer
verifier on detecting the v5 superblock version. And the CRC of the
buffer is displayed as being correct as the verifier didn't throw a
corruption error when it ran.

This output currently causes all sorts of problems for xfstests -
the root inode cluster is another that causes issues, and there are
also cases occurring where cache purging on unmount is throwing
warnings. Hence it's not yet ready for merge, but otherwise it seems
to work reliably enough to use. Early review and testing would be
helpful at this point.

These changes also mean that we should be able to turn metadump back
on, as it needs CRC write support for the obfuscation code to
work on v5 filesystems. Patches will be coming shortly to do that.

xfs_check is still an open question. I might just re-enable it and
add CRC validation checks and see what falls out from that, without
adding any extra validation for the v5 filesystem format bits...

Comments, thoughts, flames?

Cheers,

Dave.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 13+ messages in thread
* [RFC PATCH 01/12] db: separate out straight buffer IO from map based IO.
  2013-09-05 12:13 [RFC PATCH 00/12] xfs_db: write support for v5 filesystems Dave Chinner
@ 2013-09-05 12:13 ` Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 02/12] db: rewrite bbmap to use xfs_buf_map Dave Chinner
                   ` (10 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2013-09-05 12:13 UTC (permalink / raw)
  To: xfs

From: Dave Chinner <dchinner@redhat.com>

First step in converting to libxfs based IO.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 db/init.c |   7 ++--
 db/io.c   | 125 +++++++++++++++++++++++++++++++++++++++++++++++++-------------
 db/io.h   |   5 +--
 3 files changed, 104 insertions(+), 33 deletions(-)

diff --git a/db/init.c b/db/init.c
index 2932e51..0e88b56 100644
--- a/db/init.c
+++ b/db/init.c
@@ -55,7 +55,7 @@ init(
 	char		**argv)
 {
 	xfs_sb_t	*sbp;
-	void		*bufp = NULL;
+	char		bufp[BBSIZE];
 	int		c;
 
 	setlocale(LC_ALL, "");
@@ -115,15 +115,14 @@ init(
 		exit(1);
 	}
 
-	if (read_bbs(XFS_SB_DADDR, 1, &bufp, NULL)) {
+	if (read_buf(XFS_SB_DADDR, 1, bufp)) {
 		fprintf(stderr, _("%s: %s is invalid (cannot read first 512 "
 			"bytes)\n"), progname, fsdevice);
 		exit(1);
 	}
 
 	/* copy SB from buffer to in-core, converting architecture as we go */
-	libxfs_sb_from_disk(&xmount.m_sb, bufp);
-	xfree(bufp);
+	libxfs_sb_from_disk(&xmount.m_sb, (struct xfs_dsb *)bufp);
 
 	sbp = &xmount.m_sb;
 	if (sbp->sb_magicnum != XFS_SB_MAGIC) {
diff --git a/db/io.c b/db/io.c
index 39a1827..fa11646 100644
--- a/db/io.c
+++ b/db/io.c
@@ -417,8 +417,61 @@ ring_add(void)
 	}
 }
 
-
 int
+read_buf(
+	xfs_daddr_t	bbno,
+	int		count,
+	void		*bufp)
+{
+	int		err;
+
+	err = pread64(x.dfd, bufp, BBTOB(count), BBTOB(bbno));
+	if (err < 0)
+		err = errno;
+	else if (err < count)
+		err = -1;
+	return err;
+}
+
+static int
+write_buf(
+	xfs_daddr_t	bbno,
+	int		count,
+	void		*bufp)
+{
+	int		err;
+
+	err = pwrite64(x.dfd, bufp, BBTOB(count), BBTOB(bbno));
+	if (err < 0)
+		err = errno;
+	else if (err < count)
+		err = -1;
+	return err;
+}
+
+static void
+write_cur_buf(void)
+{
+	int ret;
+
+	ret = write_buf(iocur_top->bb, iocur_top->blen, iocur_top->buf);
+
+	if (ret == -1)
+		dbprintf(_("incomplete write, block: %lld\n"),
+			 (iocur_base + iocur_sp)->bb);
+	else if (ret != 0)
+		dbprintf(_("write error: %s\n"), strerror(ret));
+
+	/* re-read buffer from disk */
+	ret = read_buf(iocur_top->bb, iocur_top->blen, iocur_top->buf);
+	if (ret == -1)
+		dbprintf(_("incomplete read, block: %lld\n"),
+			 (iocur_base + iocur_sp)->bb);
+	else if (ret != 0)
+		dbprintf(_("read error: %s\n"), strerror(ret));
+}
+
+static int
 write_bbs(
 	__int64_t       bbno,
 	int             count,
@@ -430,15 +483,14 @@ write_bbs(
 	int		j;
 	int		rval = EINVAL;	/* initialize for zero `count' case */
 
-	for (j = 0; j < count; j += bbmap ? 1 : count) {
-		if (bbmap)
-			bbno = bbmap->b[j];
+	for (j = 0; j < count; j++) {
+		bbno = bbmap->b[j];
 		if (lseek64(x.dfd, bbno << BBSHIFT, SEEK_SET) < 0) {
 			rval = errno;
 			dbprintf(_("can't seek in filesystem at bb %lld\n"), bbno);
 			return rval;
 		}
-		c = BBTOB(bbmap ? 1 : count);
+		c = BBTOB(1);
 		i = (int)write(x.dfd, (char *)bufp + BBTOB(j), c);
 		if (i < 0) {
 			rval = errno;
@@ -452,7 +504,7 @@ write_bbs(
 	return rval;
 }
 
-int
+static int
 read_bbs(
 	__int64_t	bbno,
 	int		count,
@@ -473,9 +525,8 @@ read_bbs(
 		buf = xmalloc(c);
 	else
 		buf = *bufp;
-	for (j = 0; j < count; j += bbmap ? 1 : count) {
-		if (bbmap)
-			bbno = bbmap->b[j];
+	for (j = 0; j < count; j++) {
+		bbno = bbmap->b[j];
 		if (lseek64(x.dfd, bbno << BBSHIFT, SEEK_SET) < 0) {
 			rval = errno;
 			dbprintf(_("can't seek in filesystem at bb %lld\n"), bbno);
@@ -483,7 +534,7 @@ read_bbs(
 				xfree(buf);
 			buf = NULL;
 		} else {
-			c = BBTOB(bbmap ? 1 : count);
+			c = BBTOB(1);
 			i = (int)read(x.dfd, (char *)buf + BBTOB(j), c);
 			if (i < 0) {
 				rval = errno;
@@ -506,22 +557,19 @@ read_bbs(
 	return rval;
 }
 
-void
-write_cur(void)
+static void
+write_cur_bbs(void)
 {
 	int ret;
 
-	if (iocur_sp < 0) {
-		dbprintf(_("nothing to write\n"));
-		return;
-	}
 	ret = write_bbs(iocur_top->bb, iocur_top->blen, iocur_top->buf,
-		iocur_top->use_bbmap ? &iocur_top->bbmap : NULL);
+			&iocur_top->bbmap);
 	if (ret == -1)
 		dbprintf(_("incomplete write, block: %lld\n"),
 			 (iocur_base + iocur_sp)->bb);
 	else if (ret != 0)
 		dbprintf(_("write error: %s\n"), strerror(ret));
+
 	/* re-read buffer from disk */
 	ret = read_bbs(iocur_top->bb, iocur_top->blen, &iocur_top->buf,
 		iocur_top->use_bbmap ? &iocur_top->bbmap : NULL);
@@ -533,6 +581,20 @@ write_cur(void)
 }
 
 void
+write_cur(void)
+{
+	if (iocur_sp < 0) {
+		dbprintf(_("nothing to write\n"));
+		return;
+	}
+
+	if (iocur_top->use_bbmap)
+		write_cur_bbs();
+	else
+		write_cur_buf();
+}
+
+void
 set_cur(
 	const typ_t	*t,
 	__int64_t	d,
@@ -549,17 +611,32 @@ set_cur(
 		return;
 	}
 
-#ifdef DEBUG
-	if (bbmap)
-		printf(_("xfs_db got a bbmap for %lld\n"), (long long)d);
-#endif
 	ino = iocur_top->ino;
 	dirino = iocur_top->dirino;
 	mode = iocur_top->mode;
 	pop_cur();
 	push_cur();
-	if (read_bbs(d, c, &iocur_top->buf, bbmap))
-		return;
+
+	if (bbmap) {
+#ifdef DEBUG
+		printf(_("xfs_db got a bbmap for %lld\n"), (long long)d);
+#endif
+
+		if (read_bbs(d, c, &iocur_top->buf, bbmap))
+			return;
+		iocur_top->bbmap = *bbmap;
+		iocur_top->use_bbmap = 1;
+	} else {
+		if (!iocur_top->buf) {
+			iocur_top->buf = malloc(BBTOB(c));
+			if (!iocur_top->buf)
+				return;
+		}
+		if (read_buf(d, c, iocur_top->buf))
+			return;
+		iocur_top->use_bbmap = 0;
+	}
+
 	iocur_top->bb = d;
 	iocur_top->blen = c;
 	iocur_top->boff = 0;
@@ -570,8 +647,6 @@ set_cur(
 	iocur_top->ino = ino;
 	iocur_top->dirino = dirino;
 	iocur_top->mode = mode;
-	if ((iocur_top->use_bbmap = (bbmap != NULL)))
-		iocur_top->bbmap = *bbmap;
 
 	/* store location in ring */
 	if (ring_flag)
diff --git a/db/io.h b/db/io.h
index 549aad9..9ea6223 100644
--- a/db/io.h
+++ b/db/io.h
@@ -52,10 +52,7 @@ extern void	off_cur(int off, int len);
 extern void	pop_cur(void);
 extern void	print_iocur(char *tag, iocur_t *ioc);
 extern void	push_cur(void);
-extern int	read_bbs(__int64_t daddr, int count, void **bufp,
-			 bbmap_t *bbmap);
-extern int	write_bbs(__int64_t daddr, int count, void *bufp,
-			  bbmap_t *bbmap);
+extern int	read_buf(__int64_t daddr, int count, void *bufp);
 extern void     write_cur(void);
 extern void	set_cur(const struct typ *t, __int64_t d, int c, int ring_add,
 			bbmap_t *bbmap);
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [RFC PATCH 02/12] db: rewrite bbmap to use xfs_buf_map
  2013-09-05 12:13 [RFC PATCH 00/12] xfs_db: write support for v5 filesystems Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 01/12] db: separate out straight buffer IO from map based IO Dave Chinner
@ 2013-09-05 12:13 ` Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 03/12] db: rewrite IO engine to use libxfs Dave Chinner
                   ` (9 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2013-09-05 12:13 UTC (permalink / raw)
  To: xfs

From: Dave Chinner <dchinner@redhat.com>

Use the libxfs struct xfs_buf_map for recording the extent layout of
discontiguous buffers and convert the read/write to decode them
directory and use read_buf/write_buf to do the extent IO. This
brings the physical xfs_db IO code to be very close to the model
that libxfs uses.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 db/bmap.c | 14 +++-----------
 db/io.c   | 58 ++++++++++++----------------------------------------------
 db/io.h   |  2 +-
 3 files changed, 16 insertions(+), 58 deletions(-)

diff --git a/db/bmap.c b/db/bmap.c
index 0ef7a62..eb5db66 100644
--- a/db/bmap.c
+++ b/db/bmap.c
@@ -293,19 +293,11 @@ make_bbmap(
 	int		nex,
 	bmap_ext_t	*bmp)
 {
-	int		d;
-	xfs_dfsbno_t	dfsbno;
 	int		i;
-	int		j;
-	int		k;
 
-	for (i = 0, d = 0; i < nex; i++) {
-		dfsbno = bmp[i].startblock;
-		for (j = 0; j < bmp[i].blockcount; j++, dfsbno++) {
-			for (k = 0; k < blkbb; k++)
-				bbmap->b[d++] =
-					XFS_FSB_TO_DADDR(mp, dfsbno) + k;
-		}
+	for (i = 0; i < nex; i++) {
+		bbmap->b[i].bm_bn = XFS_FSB_TO_DADDR(mp, bmp[i].startblock);
+		bbmap->b[i].bm_bn = XFS_FSB_TO_BB(mp, bmp[i].blockcount);
 	}
 }
 
diff --git a/db/io.c b/db/io.c
index fa11646..01a5970 100644
--- a/db/io.c
+++ b/db/io.c
@@ -478,28 +478,16 @@ write_bbs(
 	void            *bufp,
 	bbmap_t		*bbmap)
 {
-	int		c;
-	int		i;
 	int		j;
 	int		rval = EINVAL;	/* initialize for zero `count' case */
 
-	for (j = 0; j < count; j++) {
-		bbno = bbmap->b[j];
-		if (lseek64(x.dfd, bbno << BBSHIFT, SEEK_SET) < 0) {
-			rval = errno;
-			dbprintf(_("can't seek in filesystem at bb %lld\n"), bbno);
-			return rval;
-		}
-		c = BBTOB(1);
-		i = (int)write(x.dfd, (char *)bufp + BBTOB(j), c);
-		if (i < 0) {
-			rval = errno;
-		} else if (i < c) {
-			rval = -1;
-		} else
-			rval = 0;
+	for (j = 0; j < count;) {
+		rval = write_buf(bbmap->b[j].bm_bn, bbmap->b[j].bm_len,
+			     (char *)bufp + BBTOB(j));
 		if (rval)
 			break;
+
+		j += bbmap->b[j].bm_len;
 	}
 	return rval;
 }
@@ -512,45 +500,23 @@ read_bbs(
 	bbmap_t		*bbmap)
 {
 	void		*buf;
-	int		c;
-	int		i;
 	int		j;
 	int		rval = EINVAL;
 
 	if (count <= 0)
 		count = 1;
 
-	c = BBTOB(count);
 	if (*bufp == NULL)
-		buf = xmalloc(c);
+		buf = xmalloc(BBTOB(count));
 	else
 		buf = *bufp;
-	for (j = 0; j < count; j++) {
-		bbno = bbmap->b[j];
-		if (lseek64(x.dfd, bbno << BBSHIFT, SEEK_SET) < 0) {
-			rval = errno;
-			dbprintf(_("can't seek in filesystem at bb %lld\n"), bbno);
-			if (*bufp == NULL)
-				xfree(buf);
-			buf = NULL;
-		} else {
-			c = BBTOB(1);
-			i = (int)read(x.dfd, (char *)buf + BBTOB(j), c);
-			if (i < 0) {
-				rval = errno;
-				if (*bufp == NULL)
-					xfree(buf);
-				buf = NULL;
-			} else if (i < c) {
-				rval = -1;
-				if (*bufp == NULL)
-					xfree(buf);
-				buf = NULL;
-			} else
-				rval = 0;
-		}
-		if (buf == NULL)
+	for (j = 0; j < count;) {
+		rval = read_buf(bbmap->b[j].bm_bn, bbmap->b[j].bm_len,
+			     (char *)buf + BBTOB(j));
+		if (rval)
 			break;
+
+		j += bbmap->b[j].bm_len;
 	}
 	if (*bufp == NULL)
 		*bufp = buf;
diff --git a/db/io.h b/db/io.h
index 9ea6223..0f7c018 100644
--- a/db/io.h
+++ b/db/io.h
@@ -20,7 +20,7 @@ struct typ;
 
 #define	BBMAP_SIZE		(XFS_MAX_BLOCKSIZE / BBSIZE)
 typedef struct bbmap {
-	__int64_t		b[BBMAP_SIZE];
+	struct xfs_buf_map	b[BBMAP_SIZE];
 } bbmap_t;
 
 typedef struct iocur {
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [RFC PATCH 03/12] db: rewrite IO engine to use libxfs
  2013-09-05 12:13 [RFC PATCH 00/12] xfs_db: write support for v5 filesystems Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 01/12] db: separate out straight buffer IO from map based IO Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 02/12] db: rewrite bbmap to use xfs_buf_map Dave Chinner
@ 2013-09-05 12:13 ` Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 04/12] xfs: split dquot buffer operations out Dave Chinner
                   ` (8 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2013-09-05 12:13 UTC (permalink / raw)
  To: xfs

From: Dave Chinner <dchinner@redhat.com>

Now that we have buffers and xfs_buf_maps, it is relatively easy to
convert the IO engine to use libxfs routines. This gets rid of the
most of the differences between mapped and straight buffer reads,
and tracks xfs_bufs directly in the IO context that is being used.

This is not yet a perfect solution, as xfs_db does different sized
IOs for the same block range which will throw warnings like:

xfs_db> inode 64
7ffff7fde740: Badness in key lookup (length)
bp=(bno 0x40, len 8192 bytes) key=(bno 0x40, len 4096 bytes)
xfs_db>

This is when first displaying an inode in the root inode chunk.
These will need to be dealt with on a case by case basis.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 db/bmap.c        |   1 +
 db/init.c        |  25 ++++++--
 db/io.c          | 170 +++++++++++++------------------------------------------
 db/io.h          |   5 +-
 include/libxfs.h |   3 +
 libxfs/init.c    |   5 ++
 libxfs/rdwr.c    |  61 +++++++++++++-------
 7 files changed, 112 insertions(+), 158 deletions(-)

diff --git a/db/bmap.c b/db/bmap.c
index eb5db66..4ac9632 100644
--- a/db/bmap.c
+++ b/db/bmap.c
@@ -299,6 +299,7 @@ make_bbmap(
 		bbmap->b[i].bm_bn = XFS_FSB_TO_DADDR(mp, bmp[i].startblock);
 		bbmap->b[i].bm_bn = XFS_FSB_TO_BB(mp, bmp[i].blockcount);
 	}
+	bbmap->nmaps = nex;
 }
 
 static xfs_fsblock_t
diff --git a/db/init.c b/db/init.c
index 0e88b56..b1cddca 100644
--- a/db/init.c
+++ b/db/init.c
@@ -54,8 +54,8 @@ init(
 	int		argc,
 	char		**argv)
 {
-	xfs_sb_t	*sbp;
-	char		bufp[BBSIZE];
+	struct xfs_sb	*sbp;
+	struct xfs_buf	*bp;
 	int		c;
 
 	setlocale(LC_ALL, "");
@@ -115,14 +115,25 @@ init(
 		exit(1);
 	}
 
-	if (read_buf(XFS_SB_DADDR, 1, bufp)) {
+	/*
+	 * Read the superblock, but don't validate it - we are a diagnostic
+	 * tool and so need to be able to mount busted filesystems.
+	 */
+	memset(&xmount, 0, sizeof(struct xfs_mount));
+	libxfs_buftarg_init(&xmount, x.ddev, x.logdev, x.rtdev);
+	bp = libxfs_readbuf(xmount.m_ddev_targp, XFS_SB_DADDR,
+			    1 << (XFS_MAX_SECTORSIZE_LOG - BBSHIFT), 0, NULL);
+
+	if (!bp || bp->b_error) {
 		fprintf(stderr, _("%s: %s is invalid (cannot read first 512 "
 			"bytes)\n"), progname, fsdevice);
 		exit(1);
 	}
 
 	/* copy SB from buffer to in-core, converting architecture as we go */
-	libxfs_sb_from_disk(&xmount.m_sb, (struct xfs_dsb *)bufp);
+	libxfs_sb_from_disk(&xmount.m_sb, XFS_BUF_TO_SBP(bp));
+	libxfs_putbuf(bp);
+	libxfs_purgebuf(bp);
 
 	sbp = &xmount.m_sb;
 	if (sbp->sb_magicnum != XFS_SB_MAGIC) {
@@ -201,6 +212,12 @@ main(
 	}
 
 close_devices:
+	/*
+	 * make sure that we pop the last buffer context we held so that the
+	 * buffer is released before purge the caches during unmount.
+	 */
+	pop_cur();
+	libxfs_umount(mp);
 	if (x.ddev)
 		libxfs_device_close(x.ddev);
 	if (x.logdev && x.logdev != x.ddev)
diff --git a/db/io.c b/db/io.c
index 01a5970..abfbca4 100644
--- a/db/io.c
+++ b/db/io.c
@@ -104,8 +104,12 @@ pop_cur(void)
 		dbprintf(_("can't pop anything from I/O stack\n"));
 		return;
 	}
-	if (iocur_top->buf)
-		xfree(iocur_top->buf);
+	if (iocur_top->bp)
+		libxfs_putbuf(iocur_top->bp);
+	if (iocur_top->bbmap) {
+		free(iocur_top->bbmap);
+		iocur_top->bbmap = NULL;
+	}
 	if (--iocur_sp >= 0) {
 		iocur_top = iocur_base + iocur_sp;
 		cur_typ = iocur_top->typ;
@@ -147,10 +151,11 @@ print_iocur(
 	dbprintf(_("\tbuffer block %lld (fsbno %lld), %d bb%s\n"), ioc->bb,
 		(xfs_dfsbno_t)XFS_DADDR_TO_FSB(mp, ioc->bb), ioc->blen,
 		ioc->blen == 1 ? "" : "s");
-	if (ioc->use_bbmap) {
+	if (ioc->bbmap) {
 		dbprintf(_("\tblock map"));
-		for (i = 0; i < ioc->blen; i++)
-			dbprintf(" %d:%lld", i, ioc->bbmap.b[i]);
+		for (i = 0; i < ioc->bbmap->nmaps; i++)
+			dbprintf(" %lld:%d", ioc->bbmap->b[i].bm_bn,
+					     ioc->bbmap->b[i].bm_len);
 		dbprintf("\n");
 	}
 	dbprintf(_("\tinode %lld, dir inode %lld, type %s\n"), ioc->ino,
@@ -238,7 +243,7 @@ push_f(
 	else
 		set_cur(iocur_top[-1].typ, iocur_top[-1].bb,
 			iocur_top[-1].blen, DB_RING_IGN,
-			iocur_top[-1].use_bbmap ? &iocur_top[-1].bbmap : NULL);
+			iocur_top[-1].bbmap);
 
 	/* run requested command */
 	if (argc>1)
@@ -280,8 +285,7 @@ forward_f(
 		iocur_ring[ring_current].bb,
 		iocur_ring[ring_current].blen,
 		DB_RING_IGN,
-		iocur_ring[ring_current].use_bbmap ?
-			&iocur_ring[ring_current].bbmap : NULL);
+		iocur_ring[ring_current].bbmap);
 
 	return 0;
 }
@@ -321,8 +325,7 @@ back_f(
 		iocur_ring[ring_current].bb,
 		iocur_ring[ring_current].blen,
 		DB_RING_IGN,
-		iocur_ring[ring_current].use_bbmap ?
-			&iocur_ring[ring_current].bbmap : NULL);
+		iocur_ring[ring_current].bbmap);
 
 	return 0;
 }
@@ -362,7 +365,7 @@ ring_f(
 		iocur_ring[index].bb,
 		iocur_ring[index].blen,
 		DB_RING_IGN,
-		iocur_ring[index].use_bbmap ? &iocur_ring[index].bbmap : NULL);
+		iocur_ring[index].bbmap);
 
 	return 0;
 }
@@ -417,132 +420,37 @@ ring_add(void)
 	}
 }
 
-int
-read_buf(
-	xfs_daddr_t	bbno,
-	int		count,
-	void		*bufp)
-{
-	int		err;
-
-	err = pread64(x.dfd, bufp, BBTOB(count), BBTOB(bbno));
-	if (err < 0)
-		err = errno;
-	else if (err < count)
-		err = -1;
-	return err;
-}
-
-static int
-write_buf(
-	xfs_daddr_t	bbno,
-	int		count,
-	void		*bufp)
-{
-	int		err;
-
-	err = pwrite64(x.dfd, bufp, BBTOB(count), BBTOB(bbno));
-	if (err < 0)
-		err = errno;
-	else if (err < count)
-		err = -1;
-	return err;
-}
-
 static void
 write_cur_buf(void)
 {
 	int ret;
 
-	ret = write_buf(iocur_top->bb, iocur_top->blen, iocur_top->buf);
-
-	if (ret == -1)
-		dbprintf(_("incomplete write, block: %lld\n"),
-			 (iocur_base + iocur_sp)->bb);
-	else if (ret != 0)
+	ret = libxfs_writebufr(iocur_top->bp);
+	if (ret != 0)
 		dbprintf(_("write error: %s\n"), strerror(ret));
 
 	/* re-read buffer from disk */
-	ret = read_buf(iocur_top->bb, iocur_top->blen, iocur_top->buf);
-	if (ret == -1)
-		dbprintf(_("incomplete read, block: %lld\n"),
-			 (iocur_base + iocur_sp)->bb);
-	else if (ret != 0)
+	ret = libxfs_readbufr(mp->m_ddev_targp, iocur_top->bb, iocur_top->bp,
+			      iocur_top->blen, 0);
+	if (ret != 0)
 		dbprintf(_("read error: %s\n"), strerror(ret));
 }
 
-static int
-write_bbs(
-	__int64_t       bbno,
-	int             count,
-	void            *bufp,
-	bbmap_t		*bbmap)
-{
-	int		j;
-	int		rval = EINVAL;	/* initialize for zero `count' case */
-
-	for (j = 0; j < count;) {
-		rval = write_buf(bbmap->b[j].bm_bn, bbmap->b[j].bm_len,
-			     (char *)bufp + BBTOB(j));
-		if (rval)
-			break;
-
-		j += bbmap->b[j].bm_len;
-	}
-	return rval;
-}
-
-static int
-read_bbs(
-	__int64_t	bbno,
-	int		count,
-	void		**bufp,
-	bbmap_t		*bbmap)
-{
-	void		*buf;
-	int		j;
-	int		rval = EINVAL;
-
-	if (count <= 0)
-		count = 1;
-
-	if (*bufp == NULL)
-		buf = xmalloc(BBTOB(count));
-	else
-		buf = *bufp;
-	for (j = 0; j < count;) {
-		rval = read_buf(bbmap->b[j].bm_bn, bbmap->b[j].bm_len,
-			     (char *)buf + BBTOB(j));
-		if (rval)
-			break;
-
-		j += bbmap->b[j].bm_len;
-	}
-	if (*bufp == NULL)
-		*bufp = buf;
-	return rval;
-}
-
 static void
 write_cur_bbs(void)
 {
 	int ret;
 
-	ret = write_bbs(iocur_top->bb, iocur_top->blen, iocur_top->buf,
-			&iocur_top->bbmap);
-	if (ret == -1)
-		dbprintf(_("incomplete write, block: %lld\n"),
-			 (iocur_base + iocur_sp)->bb);
-	else if (ret != 0)
+	ret = libxfs_writebufr(iocur_top->bp);
+	if (ret != 0)
 		dbprintf(_("write error: %s\n"), strerror(ret));
 
+
 	/* re-read buffer from disk */
-	ret = read_bbs(iocur_top->bb, iocur_top->blen, &iocur_top->buf,
-		iocur_top->use_bbmap ? &iocur_top->bbmap : NULL);
-	if (ret == -1)
-		dbprintf(_("incomplete read, block: %lld\n"),
-			 (iocur_base + iocur_sp)->bb);
-	else if (ret != 0)
+	ret = libxfs_readbufr_map(mp->m_ddev_targp, iocur_top->bp,
+				  iocur_top->bbmap->b, iocur_top->bbmap->nmaps,
+				  0);
+	if (ret != 0)
 		dbprintf(_("read error: %s\n"), strerror(ret));
 }
 
@@ -554,7 +462,7 @@ write_cur(void)
 		return;
 	}
 
-	if (iocur_top->use_bbmap)
+	if (iocur_top->bbmap)
 		write_cur_bbs();
 	else
 		write_cur_buf();
@@ -568,6 +476,7 @@ set_cur(
 	int             ring_flag,
 	bbmap_t		*bbmap)
 {
+	struct xfs_buf	*bp;
 	xfs_ino_t	dirino;
 	xfs_ino_t	ino;
 	__uint16_t	mode;
@@ -587,21 +496,20 @@ set_cur(
 #ifdef DEBUG
 		printf(_("xfs_db got a bbmap for %lld\n"), (long long)d);
 #endif
-
-		if (read_bbs(d, c, &iocur_top->buf, bbmap))
+		iocur_top->bbmap = malloc(sizeof(struct bbmap));
+		if (!iocur_top->bbmap)
 			return;
-		iocur_top->bbmap = *bbmap;
-		iocur_top->use_bbmap = 1;
+		memcpy(iocur_top->bbmap, bbmap, sizeof(struct bbmap));
+		bp = libxfs_readbuf_map(mp->m_ddev_targp, bbmap->b,
+					bbmap->nmaps, 0, NULL);
 	} else {
-		if (!iocur_top->buf) {
-			iocur_top->buf = malloc(BBTOB(c));
-			if (!iocur_top->buf)
-				return;
-		}
-		if (read_buf(d, c, iocur_top->buf))
-			return;
-		iocur_top->use_bbmap = 0;
+		bp = libxfs_readbuf(mp->m_ddev_targp, d, c, 0, NULL);
+		iocur_top->bbmap = NULL;
 	}
+	if (!bp || bp->b_error)
+		return;
+	iocur_top->buf = bp->b_addr;
+	iocur_top->bp = bp;
 
 	iocur_top->bb = d;
 	iocur_top->blen = c;
diff --git a/db/io.h b/db/io.h
index 0f7c018..2c47ccc 100644
--- a/db/io.h
+++ b/db/io.h
@@ -20,6 +20,7 @@ struct typ;
 
 #define	BBMAP_SIZE		(XFS_MAX_BLOCKSIZE / BBSIZE)
 typedef struct bbmap {
+	int			nmaps;
 	struct xfs_buf_map	b[BBMAP_SIZE];
 } bbmap_t;
 
@@ -35,8 +36,8 @@ typedef struct iocur {
 	__uint16_t		mode;	/* current inode's mode */
 	xfs_off_t		off;	/* fs offset of "data" in bytes */
 	const struct typ	*typ;	/* type of "data" */
-	int			use_bbmap; /* set if bbmap is valid */
-	bbmap_t			bbmap;	/* map daddr if fragmented */
+	bbmap_t			*bbmap;	/* map daddr if fragmented */
+	struct xfs_buf		*bp;	/* underlying buffer */
 } iocur_t;
 
 #define DB_RING_ADD 1                   /* add to ring on set_cur */
diff --git a/include/libxfs.h b/include/libxfs.h
index ffd83bf..bee0663 100644
--- a/include/libxfs.h
+++ b/include/libxfs.h
@@ -437,7 +437,10 @@ extern xfs_buf_t *libxfs_getbufr(struct xfs_buftarg *, xfs_daddr_t, int);
 extern void	libxfs_putbufr(xfs_buf_t *);
 
 extern int	libxfs_writebuf_int(xfs_buf_t *, int);
+extern int	libxfs_writebufr(struct xfs_buf *);
 extern int	libxfs_readbufr(struct xfs_buftarg *, xfs_daddr_t, xfs_buf_t *, int, int);
+extern int	libxfs_readbufr_map(struct xfs_buftarg *, struct xfs_buf *,
+				    struct xfs_buf_map *, int, int);
 
 extern int libxfs_bhash_size;
 extern int libxfs_ihash_size;
diff --git a/libxfs/init.c b/libxfs/init.c
index db7eeea..229aa50 100644
--- a/libxfs/init.c
+++ b/libxfs/init.c
@@ -688,6 +688,9 @@ libxfs_mount(
 	libxfs_buftarg_init(mp, dev, logdev, rtdev);
 
 	mp->m_flags = (LIBXFS_MOUNT_32BITINODES|LIBXFS_MOUNT_32BITINOOPT);
+	if (flags & LIBXFS_MOUNT_ROOTINOS)
+		mp->m_flags |= LIBXFS_MOUNT_ROOTINOS;
+
 	mp->m_sb = *sb;
 	INIT_RADIX_TREE(&mp->m_perag_tree, GFP_KERNEL);
 	sbp = &(mp->m_sb);
@@ -866,6 +869,8 @@ libxfs_umount(xfs_mount_t *mp)
 	int			agno;
 
 	libxfs_rtmount_destroy(mp);
+	if ((mp->m_flags & LIBXFS_MOUNT_ROOTINOS) && mp->m_rootip)
+		libxfs_iput(mp->m_rootip, 0);
 	libxfs_icache_purge();
 	libxfs_bcache_purge();
 
diff --git a/libxfs/rdwr.c b/libxfs/rdwr.c
index 13dbd23..8d8bcfc 100644
--- a/libxfs/rdwr.c
+++ b/libxfs/rdwr.c
@@ -719,30 +719,18 @@ libxfs_readbuf(struct xfs_buftarg *btp, xfs_daddr_t blkno, int len, int flags,
 	return bp;
 }
 
-struct xfs_buf *
-libxfs_readbuf_map(struct xfs_buftarg *btp, struct xfs_buf_map *map, int nmaps,
-		int flags, const struct xfs_buf_ops *ops)
+int
+libxfs_readbufr_map(struct xfs_buftarg *btp, struct xfs_buf *bp,
+		    struct xfs_buf_map *map, int nmaps, int flags)
 {
-	xfs_buf_t	*bp;
-	int		error = 0;
-	int		fd;
-	int		i;
-	char		*buf;
-
-	if (nmaps == 1)
-		return libxfs_readbuf(btp, map[0].bm_bn, map[0].bm_len,
-					flags, ops);
-
-	bp = libxfs_getbuf_map(btp, map, nmaps);
-	if (!bp)
-		return NULL;
+	int	fd = libxfs_device_to_fd(btp->dev);
+	int	error = 0;
+	char	*buf;
+	int	i;
 
-	bp->b_error = 0;
-	bp->b_ops = ops;
-	if ((bp->b_flags & (LIBXFS_B_UPTODATE|LIBXFS_B_DIRTY)))
-		return bp;
+	ASSERT(BBTOB(len) <= bp->b_bcount);
 
-	ASSERT(bp->b_nmaps = nmaps);
+	ASSERT(bp->b_nmaps == nmaps);
 
 	fd = libxfs_device_to_fd(btp->dev);
 	buf = bp->b_addr;
@@ -762,6 +750,37 @@ libxfs_readbuf_map(struct xfs_buftarg *btp, struct xfs_buf_map *map, int nmaps,
 		offset += len;
 	}
 
+	if (!error);
+		bp->b_flags |= LIBXFS_B_UPTODATE;
+#ifdef IO_DEBUG
+	printf("%lx: %s: read %u bytes, error %d, blkno=0x%llx(0x%llx), %p\n",
+		pthread_self(), __FUNCTION__, , error,
+		(long long)LIBXFS_BBTOOFF64(blkno), (long long)blkno, bp);
+#endif
+	return error;
+}
+
+struct xfs_buf *
+libxfs_readbuf_map(struct xfs_buftarg *btp, struct xfs_buf_map *map, int nmaps,
+		int flags, const struct xfs_buf_ops *ops)
+{
+	struct xfs_buf	*bp;
+	int		error = 0;
+
+	if (nmaps == 1)
+		return libxfs_readbuf(btp, map[0].bm_bn, map[0].bm_len,
+					flags, ops);
+
+	bp = libxfs_getbuf_map(btp, map, nmaps);
+	if (!bp)
+		return NULL;
+
+	bp->b_error = 0;
+	bp->b_ops = ops;
+	if ((bp->b_flags & (LIBXFS_B_UPTODATE|LIBXFS_B_DIRTY)))
+		return bp;
+
+	error = libxfs_readbufr_map(btp, bp, map, nmaps, flags);
 	if (!error) {
 		bp->b_flags |= LIBXFS_B_UPTODATE;
 		if (bp->b_ops)
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [RFC PATCH 04/12] xfs: split dquot buffer operations out
  2013-09-05 12:13 [RFC PATCH 00/12] xfs_db: write support for v5 filesystems Dave Chinner
                   ` (2 preceding siblings ...)
  2013-09-05 12:13 ` [RFC PATCH 03/12] db: rewrite IO engine to use libxfs Dave Chinner
@ 2013-09-05 12:13 ` Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 05/12] xfs: move buf_ops externs to a xfs_format.h Dave Chinner
                   ` (7 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2013-09-05 12:13 UTC (permalink / raw)
  To: xfs

From: Dave Chinner <dchinner@redhat.com>

Parts of userspace want to be able to read and modify dquot buffers
(e.g. xfs_db) so we need to split out the reading and writing of
these buffers so it is easy to shared code with libxfs in userspace.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 include/libxfs.h       |   9 ++
 libxfs/Makefile        |   1 +
 libxfs/xfs_dquot_buf.c | 273 +++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 283 insertions(+)
 create mode 100644 libxfs/xfs_dquot_buf.c

diff --git a/include/libxfs.h b/include/libxfs.h
index bee0663..9429f61 100644
--- a/include/libxfs.h
+++ b/include/libxfs.h
@@ -216,6 +216,15 @@ typedef struct xfs_mount {
 	xfs_dablk_t		m_dirdatablk;	/* blockno of dir data v2 */
 	xfs_dablk_t		m_dirleafblk;	/* blockno of dir non-data v2 */
 	xfs_dablk_t		m_dirfreeblk;	/* blockno of dirfreeindex v2 */
+
+	/*
+	 * anonymous struct to allow xfs_dquot_buf.c to compile.
+	 * Pointer is always null in userspace, so code does not use it at all
+	 */
+	struct {
+		int	qi_dqperchunk;
+	}			*m_quotainfo;
+
 } xfs_mount_t;
 
 /*
diff --git a/libxfs/Makefile b/libxfs/Makefile
index f0cbae3..4522218 100644
--- a/libxfs/Makefile
+++ b/libxfs/Makefile
@@ -29,6 +29,7 @@ CFILES = cache.c \
 	xfs_dir2_leaf.c \
 	xfs_dir2_node.c \
 	xfs_dir2_sf.c \
+	xfs_dquot_buf.c \
 	xfs_ialloc.c \
 	xfs_inode_buf.c \
 	xfs_inode_fork.c \
diff --git a/libxfs/xfs_dquot_buf.c b/libxfs/xfs_dquot_buf.c
new file mode 100644
index 0000000..620d9d3
--- /dev/null
+++ b/libxfs/xfs_dquot_buf.c
@@ -0,0 +1,273 @@
+/*
+ * Copyright (c) 2000-2006 Silicon Graphics, Inc.
+ * Copyright (c) 2013 Red Hat, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+ */
+#include "xfs.h"
+
+int
+xfs_calc_dquots_per_chunk(
+	struct xfs_mount	*mp,
+	unsigned int		nbblks)	/* basic block units */
+{
+	unsigned int	ndquots;
+
+	ASSERT(nbblks > 0);
+	ndquots = BBTOB(nbblks);
+	do_div(ndquots, sizeof(xfs_dqblk_t));
+
+	return ndquots;
+}
+
+/*
+ * Do some primitive error checking on ondisk dquot data structures.
+ */
+int
+xfs_dqcheck(
+	struct xfs_mount *mp,
+	xfs_disk_dquot_t *ddq,
+	xfs_dqid_t	 id,
+	uint		 type,	  /* used only when IO_dorepair is true */
+	uint		 flags,
+	char		 *str)
+{
+	xfs_dqblk_t	 *d = (xfs_dqblk_t *)ddq;
+	int		errs = 0;
+
+	/*
+	 * We can encounter an uninitialized dquot buffer for 2 reasons:
+	 * 1. If we crash while deleting the quotainode(s), and those blks got
+	 *    used for user data. This is because we take the path of regular
+	 *    file deletion; however, the size field of quotainodes is never
+	 *    updated, so all the tricks that we play in itruncate_finish
+	 *    don't quite matter.
+	 *
+	 * 2. We don't play the quota buffers when there's a quotaoff logitem.
+	 *    But the allocation will be replayed so we'll end up with an
+	 *    uninitialized quota block.
+	 *
+	 * This is all fine; things are still consistent, and we haven't lost
+	 * any quota information. Just don't complain about bad dquot blks.
+	 */
+	if (ddq->d_magic != cpu_to_be16(XFS_DQUOT_MAGIC)) {
+		if (flags & XFS_QMOPT_DOWARN)
+			xfs_alert(mp,
+			"%s : XFS dquot ID 0x%x, magic 0x%x != 0x%x",
+			str, id, be16_to_cpu(ddq->d_magic), XFS_DQUOT_MAGIC);
+		errs++;
+	}
+	if (ddq->d_version != XFS_DQUOT_VERSION) {
+		if (flags & XFS_QMOPT_DOWARN)
+			xfs_alert(mp,
+			"%s : XFS dquot ID 0x%x, version 0x%x != 0x%x",
+			str, id, ddq->d_version, XFS_DQUOT_VERSION);
+		errs++;
+	}
+
+	if (ddq->d_flags != XFS_DQ_USER &&
+	    ddq->d_flags != XFS_DQ_PROJ &&
+	    ddq->d_flags != XFS_DQ_GROUP) {
+		if (flags & XFS_QMOPT_DOWARN)
+			xfs_alert(mp,
+			"%s : XFS dquot ID 0x%x, unknown flags 0x%x",
+			str, id, ddq->d_flags);
+		errs++;
+	}
+
+	if (id != -1 && id != be32_to_cpu(ddq->d_id)) {
+		if (flags & XFS_QMOPT_DOWARN)
+			xfs_alert(mp,
+			"%s : ondisk-dquot 0x%p, ID mismatch: "
+			"0x%x expected, found id 0x%x",
+			str, ddq, id, be32_to_cpu(ddq->d_id));
+		errs++;
+	}
+
+	if (!errs && ddq->d_id) {
+		if (ddq->d_blk_softlimit &&
+		    be64_to_cpu(ddq->d_bcount) >
+				be64_to_cpu(ddq->d_blk_softlimit)) {
+			if (!ddq->d_btimer) {
+				if (flags & XFS_QMOPT_DOWARN)
+					xfs_alert(mp,
+			"%s : Dquot ID 0x%x (0x%p) BLK TIMER NOT STARTED",
+					str, (int)be32_to_cpu(ddq->d_id), ddq);
+				errs++;
+			}
+		}
+		if (ddq->d_ino_softlimit &&
+		    be64_to_cpu(ddq->d_icount) >
+				be64_to_cpu(ddq->d_ino_softlimit)) {
+			if (!ddq->d_itimer) {
+				if (flags & XFS_QMOPT_DOWARN)
+					xfs_alert(mp,
+			"%s : Dquot ID 0x%x (0x%p) INODE TIMER NOT STARTED",
+					str, (int)be32_to_cpu(ddq->d_id), ddq);
+				errs++;
+			}
+		}
+		if (ddq->d_rtb_softlimit &&
+		    be64_to_cpu(ddq->d_rtbcount) >
+				be64_to_cpu(ddq->d_rtb_softlimit)) {
+			if (!ddq->d_rtbtimer) {
+				if (flags & XFS_QMOPT_DOWARN)
+					xfs_alert(mp,
+			"%s : Dquot ID 0x%x (0x%p) RTBLK TIMER NOT STARTED",
+					str, (int)be32_to_cpu(ddq->d_id), ddq);
+				errs++;
+			}
+		}
+	}
+
+	if (!errs || !(flags & XFS_QMOPT_DQREPAIR))
+		return errs;
+
+	if (flags & XFS_QMOPT_DOWARN)
+		xfs_notice(mp, "Re-initializing dquot ID 0x%x", id);
+
+	/*
+	 * Typically, a repair is only requested by quotacheck.
+	 */
+	ASSERT(id != -1);
+	ASSERT(flags & XFS_QMOPT_DQREPAIR);
+	memset(d, 0, sizeof(xfs_dqblk_t));
+
+	d->dd_diskdq.d_magic = cpu_to_be16(XFS_DQUOT_MAGIC);
+	d->dd_diskdq.d_version = XFS_DQUOT_VERSION;
+	d->dd_diskdq.d_flags = type;
+	d->dd_diskdq.d_id = cpu_to_be32(id);
+
+	if (xfs_sb_version_hascrc(&mp->m_sb)) {
+		uuid_copy(&d->dd_uuid, &mp->m_sb.sb_uuid);
+		xfs_update_cksum((char *)d, sizeof(struct xfs_dqblk),
+				 XFS_DQUOT_CRC_OFF);
+	}
+
+	return errs;
+}
+
+STATIC bool
+xfs_dquot_buf_verify_crc(
+	struct xfs_mount	*mp,
+	struct xfs_buf		*bp)
+{
+	struct xfs_dqblk	*d = (struct xfs_dqblk *)bp->b_addr;
+	int			ndquots;
+	int			i;
+
+	if (!xfs_sb_version_hascrc(&mp->m_sb))
+		return true;
+
+	/*
+	 * if we are in log recovery, the quota subsystem has not been
+	 * initialised so we have no quotainfo structure. In that case, we need
+	 * to manually calculate the number of dquots in the buffer.
+	 */
+	if (mp->m_quotainfo)
+		ndquots = mp->m_quotainfo->qi_dqperchunk;
+	else
+		ndquots = xfs_calc_dquots_per_chunk(mp,
+					XFS_BB_TO_FSB(mp, bp->b_length));
+
+	for (i = 0; i < ndquots; i++, d++) {
+		if (!xfs_verify_cksum((char *)d, sizeof(struct xfs_dqblk),
+				 XFS_DQUOT_CRC_OFF))
+			return false;
+		if (!uuid_equal(&d->dd_uuid, &mp->m_sb.sb_uuid))
+			return false;
+	}
+	return true;
+}
+
+STATIC bool
+xfs_dquot_buf_verify(
+	struct xfs_mount	*mp,
+	struct xfs_buf		*bp)
+{
+	struct xfs_dqblk	*d = (struct xfs_dqblk *)bp->b_addr;
+	xfs_dqid_t		id = 0;
+	int			ndquots;
+	int			i;
+
+	/*
+	 * if we are in log recovery, the quota subsystem has not been
+	 * initialised so we have no quotainfo structure. In that case, we need
+	 * to manually calculate the number of dquots in the buffer.
+	 */
+	if (mp->m_quotainfo)
+		ndquots = mp->m_quotainfo->qi_dqperchunk;
+	else
+		ndquots = xfs_calc_dquots_per_chunk(mp, bp->b_length);
+
+	/*
+	 * On the first read of the buffer, verify that each dquot is valid.
+	 * We don't know what the id of the dquot is supposed to be, just that
+	 * they should be increasing monotonically within the buffer. If the
+	 * first id is corrupt, then it will fail on the second dquot in the
+	 * buffer so corruptions could point to the wrong dquot in this case.
+	 */
+	for (i = 0; i < ndquots; i++) {
+		struct xfs_disk_dquot	*ddq;
+		int			error;
+
+		ddq = &d[i].dd_diskdq;
+
+		if (i == 0)
+			id = be32_to_cpu(ddq->d_id);
+
+		error = xfs_dqcheck(mp, ddq, id + i, 0, XFS_QMOPT_DOWARN,
+				       "xfs_dquot_buf_verify");
+		if (error)
+			return false;
+	}
+	return true;
+}
+
+static void
+xfs_dquot_buf_read_verify(
+	struct xfs_buf	*bp)
+{
+	struct xfs_mount	*mp = bp->b_target->bt_mount;
+
+	if (!xfs_dquot_buf_verify_crc(mp, bp) || !xfs_dquot_buf_verify(mp, bp)) {
+		XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp, bp->b_addr);
+		xfs_buf_ioerror(bp, EFSCORRUPTED);
+	}
+}
+
+/*
+ * we don't calculate the CRC here as that is done when the dquot is flushed to
+ * the buffer after the update is done. This ensures that the dquot in the
+ * buffer always has an up-to-date CRC value.
+ */
+void
+xfs_dquot_buf_write_verify(
+	struct xfs_buf	*bp)
+{
+	struct xfs_mount	*mp = bp->b_target->bt_mount;
+
+	if (!xfs_dquot_buf_verify(mp, bp)) {
+		XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp, bp->b_addr);
+		xfs_buf_ioerror(bp, EFSCORRUPTED);
+		return;
+	}
+}
+
+const struct xfs_buf_ops xfs_dquot_buf_ops = {
+	.verify_read = xfs_dquot_buf_read_verify,
+	.verify_write = xfs_dquot_buf_write_verify,
+};
+
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [RFC PATCH 05/12] xfs: move buf_ops externs to a xfs_format.h
  2013-09-05 12:13 [RFC PATCH 00/12] xfs_db: write support for v5 filesystems Dave Chinner
                   ` (3 preceding siblings ...)
  2013-09-05 12:13 ` [RFC PATCH 04/12] xfs: split dquot buffer operations out Dave Chinner
@ 2013-09-05 12:13 ` Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 06/12] db: introduce verifier support into set_cur Dave Chinner
                   ` (6 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2013-09-05 12:13 UTC (permalink / raw)
  To: xfs

From: Dave Chinner <dchinner@redhat.com>

All of the buffer operations structures are needed to be exported
for xfs_db, so move them all to a common location rather than
spreading them all over the place. They are verifying the on-disk
format, so xfs_format.h is as good a place as any.

Don't move the dir2 buffer verifiers - they are already shared in a
"central" place for all the dir2 and moving them just requires
adding more include files...

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 include/xfs_ag.h           |  4 ----
 include/xfs_alloc.h        |  3 ---
 include/xfs_alloc_btree.h  |  2 --
 include/xfs_attr_leaf.h    |  2 --
 include/xfs_attr_remote.h  |  2 --
 include/xfs_bmap_btree.h   |  2 --
 include/xfs_da_btree.h     |  2 --
 include/xfs_format.h       | 22 ++++++++++++++++++++++
 include/xfs_ialloc.h       |  2 --
 include/xfs_ialloc_btree.h |  2 --
 include/xfs_inode_buf.h    |  2 --
 include/xfs_sb.h           |  3 ---
 12 files changed, 22 insertions(+), 26 deletions(-)

diff --git a/include/xfs_ag.h b/include/xfs_ag.h
index 1cb740a..3fc1098 100644
--- a/include/xfs_ag.h
+++ b/include/xfs_ag.h
@@ -128,8 +128,6 @@ typedef struct xfs_agf {
 extern int xfs_read_agf(struct xfs_mount *mp, struct xfs_trans *tp,
 			xfs_agnumber_t agno, int flags, struct xfs_buf **bpp);
 
-extern const struct xfs_buf_ops xfs_agf_buf_ops;
-
 /*
  * Size of the unlinked inode hash table in the agi.
  */
@@ -191,8 +189,6 @@ typedef struct xfs_agi {
 extern int xfs_read_agi(struct xfs_mount *mp, struct xfs_trans *tp,
 				xfs_agnumber_t agno, struct xfs_buf **bpp);
 
-extern const struct xfs_buf_ops xfs_agi_buf_ops;
-
 /*
  * The third a.g. block contains the a.g. freelist, an array
  * of block pointers to blocks owned by the allocation btree code.
diff --git a/include/xfs_alloc.h b/include/xfs_alloc.h
index 99d0a61..feacb06 100644
--- a/include/xfs_alloc.h
+++ b/include/xfs_alloc.h
@@ -231,7 +231,4 @@ xfs_alloc_get_rec(
 	xfs_extlen_t		*len,	/* output: length of extent */
 	int			*stat);	/* output: success/failure */
 
-extern const struct xfs_buf_ops xfs_agf_buf_ops;
-extern const struct xfs_buf_ops xfs_agfl_buf_ops;
-
 #endif	/* __XFS_ALLOC_H__ */
diff --git a/include/xfs_alloc_btree.h b/include/xfs_alloc_btree.h
index e3a3f74..72676c3 100644
--- a/include/xfs_alloc_btree.h
+++ b/include/xfs_alloc_btree.h
@@ -95,6 +95,4 @@ extern struct xfs_btree_cur *xfs_allocbt_init_cursor(struct xfs_mount *,
 		xfs_agnumber_t, xfs_btnum_t);
 extern int xfs_allocbt_maxrecs(struct xfs_mount *, int, int);
 
-extern const struct xfs_buf_ops xfs_allocbt_buf_ops;
-
 #endif	/* __XFS_ALLOC_BTREE_H__ */
diff --git a/include/xfs_attr_leaf.h b/include/xfs_attr_leaf.h
index c102213..da34e64 100644
--- a/include/xfs_attr_leaf.h
+++ b/include/xfs_attr_leaf.h
@@ -336,6 +336,4 @@ void	xfs_attr3_leaf_hdr_from_disk(struct xfs_attr3_icleaf_hdr *to,
 void	xfs_attr3_leaf_hdr_to_disk(struct xfs_attr_leafblock *to,
 				   struct xfs_attr3_icleaf_hdr *from);
 
-extern const struct xfs_buf_ops xfs_attr3_leaf_buf_ops;
-
 #endif	/* __XFS_ATTR_LEAF_H__ */
diff --git a/include/xfs_attr_remote.h b/include/xfs_attr_remote.h
index 92a8fd7..b3e1e1b 100644
--- a/include/xfs_attr_remote.h
+++ b/include/xfs_attr_remote.h
@@ -45,8 +45,6 @@ struct xfs_attr3_rmt_hdr {
 	((bufsize) - (xfs_sb_version_hascrc(&(mp)->m_sb) ? \
 			sizeof(struct xfs_attr3_rmt_hdr) : 0))
 
-extern const struct xfs_buf_ops xfs_attr3_rmt_buf_ops;
-
 int xfs_attr3_rmt_blocks(struct xfs_mount *mp, int attrlen);
 
 int xfs_attr_rmtval_get(struct xfs_da_args *args);
diff --git a/include/xfs_bmap_btree.h b/include/xfs_bmap_btree.h
index 1b726d6..e307978 100644
--- a/include/xfs_bmap_btree.h
+++ b/include/xfs_bmap_btree.h
@@ -239,6 +239,4 @@ extern int xfs_bmbt_maxrecs(struct xfs_mount *, int blocklen, int leaf);
 extern struct xfs_btree_cur *xfs_bmbt_init_cursor(struct xfs_mount *,
 		struct xfs_trans *, struct xfs_inode *, int);
 
-extern const struct xfs_buf_ops xfs_bmbt_buf_ops;
-
 #endif	/* __XFS_BMAP_BTREE_H__ */
diff --git a/include/xfs_da_btree.h b/include/xfs_da_btree.h
index b1f2679..a762cfa 100644
--- a/include/xfs_da_btree.h
+++ b/include/xfs_da_btree.h
@@ -309,8 +309,6 @@ int	xfs_da3_node_read(struct xfs_trans *tp, struct xfs_inode *dp,
 			 xfs_dablk_t bno, xfs_daddr_t mappedbno,
 			 struct xfs_buf **bpp, int which_fork);
 
-extern const struct xfs_buf_ops xfs_da3_node_buf_ops;
-
 /*
  * Utility routines.
  */
diff --git a/include/xfs_format.h b/include/xfs_format.h
index 35c08ff..c1ea4b3 100644
--- a/include/xfs_format.h
+++ b/include/xfs_format.h
@@ -34,6 +34,28 @@ struct xfs_buf;
 struct xfs_ifork;
 
 /*
+ * Central location for exporting buffer verifier operations as they are
+ * widely used, including userspace tools
+ */
+extern const struct xfs_buf_ops xfs_agf_buf_ops;
+extern const struct xfs_buf_ops xfs_agi_buf_ops;
+extern const struct xfs_buf_ops xfs_agf_buf_ops;
+extern const struct xfs_buf_ops xfs_agfl_buf_ops;
+extern const struct xfs_buf_ops xfs_allocbt_buf_ops;
+extern const struct xfs_buf_ops xfs_attr3_leaf_buf_ops;
+extern const struct xfs_buf_ops xfs_attr3_rmt_buf_ops;
+extern const struct xfs_buf_ops xfs_bmbt_buf_ops;
+extern const struct xfs_buf_ops xfs_da3_node_buf_ops;
+extern const struct xfs_buf_ops xfs_symlink_buf_ops;
+extern const struct xfs_buf_ops xfs_agi_buf_ops;
+extern const struct xfs_buf_ops xfs_inobt_buf_ops;
+extern const struct xfs_buf_ops xfs_inode_buf_ops;
+extern const struct xfs_buf_ops xfs_inode_ra_buf_ops;
+extern const struct xfs_buf_ops xfs_dquot_buf_ops;
+extern const struct xfs_buf_ops xfs_sb_buf_ops;
+extern const struct xfs_buf_ops xfs_sb_quiet_buf_ops;
+
+/*
  * RealTime Device format definitions
  */
 
diff --git a/include/xfs_ialloc.h b/include/xfs_ialloc.h
index 68c0732..1557798 100644
--- a/include/xfs_ialloc.h
+++ b/include/xfs_ialloc.h
@@ -158,6 +158,4 @@ int xfs_ialloc_inode_init(struct xfs_mount *mp, struct xfs_trans *tp,
 			  xfs_agnumber_t agno, xfs_agblock_t agbno,
 			  xfs_agblock_t length, unsigned int gen);
 
-extern const struct xfs_buf_ops xfs_agi_buf_ops;
-
 #endif	/* __XFS_IALLOC_H__ */
diff --git a/include/xfs_ialloc_btree.h b/include/xfs_ialloc_btree.h
index 3ac36b7..cfbfe46 100644
--- a/include/xfs_ialloc_btree.h
+++ b/include/xfs_ialloc_btree.h
@@ -110,6 +110,4 @@ extern struct xfs_btree_cur *xfs_inobt_init_cursor(struct xfs_mount *,
 		struct xfs_trans *, struct xfs_buf *, xfs_agnumber_t);
 extern int xfs_inobt_maxrecs(struct xfs_mount *, int, int);
 
-extern const struct xfs_buf_ops xfs_inobt_buf_ops;
-
 #endif	/* __XFS_IALLOC_BTREE_H__ */
diff --git a/include/xfs_inode_buf.h b/include/xfs_inode_buf.h
index aae9fc4..e8fd3bd 100644
--- a/include/xfs_inode_buf.h
+++ b/include/xfs_inode_buf.h
@@ -47,6 +47,4 @@ void		xfs_inobp_check(struct xfs_mount *, struct xfs_buf *);
 #define	xfs_inobp_check(mp, bp)
 #endif /* DEBUG */
 
-extern const struct xfs_buf_ops xfs_inode_buf_ops;
-
 #endif	/* __XFS_INODE_BUF_H__ */
diff --git a/include/xfs_sb.h b/include/xfs_sb.h
index 1d0839a..bc62812 100644
--- a/include/xfs_sb.h
+++ b/include/xfs_sb.h
@@ -693,7 +693,4 @@ extern void	xfs_sb_from_disk(struct xfs_sb *, struct xfs_dsb *);
 extern void	xfs_sb_to_disk(struct xfs_dsb *, struct xfs_sb *, __int64_t);
 extern void	xfs_sb_quota_from_disk(struct xfs_sb *sbp);
 
-extern const struct xfs_buf_ops xfs_sb_buf_ops;
-extern const struct xfs_buf_ops xfs_sb_quiet_buf_ops;
-
 #endif	/* __XFS_SB_H__ */
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [RFC PATCH 06/12] db: introduce verifier support into set_cur
  2013-09-05 12:13 [RFC PATCH 00/12] xfs_db: write support for v5 filesystems Dave Chinner
                   ` (4 preceding siblings ...)
  2013-09-05 12:13 ` [RFC PATCH 05/12] xfs: move buf_ops externs to a xfs_format.h Dave Chinner
@ 2013-09-05 12:13 ` Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 07/12] db: indicate if the CRC on a buffer is correct or not Dave Chinner
                   ` (5 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2013-09-05 12:13 UTC (permalink / raw)
  To: xfs

From: Dave Chinner <dchinner@redhat.com>

To be able to use read and write verifiers, we need to pass the
verifier to the IO routines. We do this via the set_cur() function
used to trigger reading the buffer.

For most metadata types, there is only one type of verifier needed.
For these, we can simply add the verifier to the type table entry
for the given type and use that directly. This type entry is already
carried around by the IO context, so if we ever need to get it again
we have direct access to it in the context we'll be doing IO.

Only attach the verifiers to the v5 filesystem type table; there is
not need for them on v4 filesystems as we don't have to verify or
calculate CRCs for them.

There are some metadata types that have more than one buffer format,
or aren't based in directly in buffers. For these, leave the type
table verifier NULL for now - these will need to be addressed
individually.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 db/io.c   | 13 +++++++---
 db/type.c | 88 ++++++++++++++++++++++++++++++++++-----------------------------
 db/type.h |  1 +
 3 files changed, 59 insertions(+), 43 deletions(-)

diff --git a/db/io.c b/db/io.c
index abfbca4..d6e1d28 100644
--- a/db/io.c
+++ b/db/io.c
@@ -480,12 +480,14 @@ set_cur(
 	xfs_ino_t	dirino;
 	xfs_ino_t	ino;
 	__uint16_t	mode;
+	const struct xfs_buf_ops *ops = t ? t->bops : NULL;
 
 	if (iocur_sp < 0) {
 		dbprintf(_("set_cur no stack element to set\n"));
 		return;
 	}
 
+
 	ino = iocur_top->ino;
 	dirino = iocur_top->dirino;
 	mode = iocur_top->mode;
@@ -501,12 +503,17 @@ set_cur(
 			return;
 		memcpy(iocur_top->bbmap, bbmap, sizeof(struct bbmap));
 		bp = libxfs_readbuf_map(mp->m_ddev_targp, bbmap->b,
-					bbmap->nmaps, 0, NULL);
+					bbmap->nmaps, 0, ops);
 	} else {
-		bp = libxfs_readbuf(mp->m_ddev_targp, d, c, 0, NULL);
+		bp = libxfs_readbuf(mp->m_ddev_targp, d, c, 0, ops);
 		iocur_top->bbmap = NULL;
 	}
-	if (!bp || bp->b_error)
+
+	/*
+	 * keep the buffer even if the verifier says it is corrupted.
+	 * We're a diagnostic tool, after all.
+	 */
+	if (!bp || (bp->b_error && bp->b_error != EFSCORRUPTED))
 		return;
 	iocur_top->buf = bp->b_addr;
 	iocur_top->bp = bp;
diff --git a/db/type.c b/db/type.c
index 64e2ef4..b3f3d87 100644
--- a/db/type.c
+++ b/db/type.c
@@ -50,50 +50,58 @@ static const cmdinfo_t	type_cmd =
 	  N_("set/show current data type"), NULL };
 
 static const typ_t	__typtab[] = {
-	{ TYP_AGF, "agf", handle_struct, agf_hfld },
-	{ TYP_AGFL, "agfl", handle_struct, agfl_hfld },
-	{ TYP_AGI, "agi", handle_struct, agi_hfld },
-	{ TYP_ATTR, "attr", handle_struct, attr_hfld },
-	{ TYP_BMAPBTA, "bmapbta", handle_struct, bmapbta_hfld },
-	{ TYP_BMAPBTD, "bmapbtd", handle_struct, bmapbtd_hfld },
-	{ TYP_BNOBT, "bnobt", handle_struct, bnobt_hfld },
-	{ TYP_CNTBT, "cntbt", handle_struct, cntbt_hfld },
-	{ TYP_DATA, "data", handle_block, NULL },
-	{ TYP_DIR2, "dir2", handle_struct, dir2_hfld },
-	{ TYP_DQBLK, "dqblk", handle_struct, dqblk_hfld },
-	{ TYP_INOBT, "inobt", handle_struct, inobt_hfld },
-	{ TYP_INODATA, "inodata", NULL, NULL },
-	{ TYP_INODE, "inode", handle_struct, inode_hfld },
-	{ TYP_LOG, "log", NULL, NULL },
-	{ TYP_RTBITMAP, "rtbitmap", NULL, NULL },
-	{ TYP_RTSUMMARY, "rtsummary", NULL, NULL },
-	{ TYP_SB, "sb", handle_struct, sb_hfld },
-	{ TYP_SYMLINK, "symlink", handle_string, NULL },
-	{ TYP_TEXT, "text", handle_text, NULL },
+	{ TYP_AGF, "agf", handle_struct, agf_hfld, NULL },
+	{ TYP_AGFL, "agfl", handle_struct, agfl_hfld, NULL },
+	{ TYP_AGI, "agi", handle_struct, agi_hfld, NULL },
+	{ TYP_ATTR, "attr", handle_struct, attr_hfld, NULL },
+	{ TYP_BMAPBTA, "bmapbta", handle_struct, bmapbta_hfld, NULL },
+	{ TYP_BMAPBTD, "bmapbtd", handle_struct, bmapbtd_hfld, NULL },
+	{ TYP_BNOBT, "bnobt", handle_struct, bnobt_hfld, NULL },
+	{ TYP_CNTBT, "cntbt", handle_struct, cntbt_hfld, NULL },
+	{ TYP_DATA, "data", handle_block, NULL, NULL },
+	{ TYP_DIR2, "dir2", handle_struct, dir2_hfld, NULL },
+	{ TYP_DQBLK, "dqblk", handle_struct, dqblk_hfld, NULL },
+	{ TYP_INOBT, "inobt", handle_struct, inobt_hfld, NULL },
+	{ TYP_INODATA, "inodata", NULL, NULL, NULL },
+	{ TYP_INODE, "inode", handle_struct, inode_hfld, NULL },
+	{ TYP_LOG, "log", NULL, NULL, NULL },
+	{ TYP_RTBITMAP, "rtbitmap", NULL, NULL, NULL },
+	{ TYP_RTSUMMARY, "rtsummary", NULL, NULL, NULL },
+	{ TYP_SB, "sb", handle_struct, sb_hfld, NULL },
+	{ TYP_SYMLINK, "symlink", handle_string, NULL, NULL },
+	{ TYP_TEXT, "text", handle_text, NULL, NULL },
 	{ TYP_NONE, NULL }
 };
 
 static const typ_t	__typtab_crc[] = {
-	{ TYP_AGF, "agf", handle_struct, agf_hfld },
-	{ TYP_AGFL, "agfl", handle_struct, agfl_crc_hfld },
-	{ TYP_AGI, "agi", handle_struct, agi_hfld },
-	{ TYP_ATTR, "attr3", handle_struct, attr3_hfld },
-	{ TYP_BMAPBTA, "bmapbta", handle_struct, bmapbta_crc_hfld },
-	{ TYP_BMAPBTD, "bmapbtd", handle_struct, bmapbtd_crc_hfld },
-	{ TYP_BNOBT, "bnobt", handle_struct, bnobt_crc_hfld },
-	{ TYP_CNTBT, "cntbt", handle_struct, cntbt_crc_hfld },
-	{ TYP_DATA, "data", handle_block, NULL },
-	{ TYP_DIR2, "dir3", handle_struct, dir3_hfld },
-	{ TYP_DQBLK, "dqblk", handle_struct, dqblk_hfld },
-	{ TYP_INOBT, "inobt", handle_struct, inobt_crc_hfld },
-	{ TYP_INODATA, "inodata", NULL, NULL },
-	{ TYP_INODE, "inode", handle_struct, inode_crc_hfld },
-	{ TYP_LOG, "log", NULL, NULL },
-	{ TYP_RTBITMAP, "rtbitmap", NULL, NULL },
-	{ TYP_RTSUMMARY, "rtsummary", NULL, NULL },
-	{ TYP_SB, "sb", handle_struct, sb_hfld },
-	{ TYP_SYMLINK, "symlink", handle_struct, symlink_crc_hfld },
-	{ TYP_TEXT, "text", handle_text, NULL },
+	{ TYP_AGF, "agf", handle_struct, agf_hfld, &xfs_agf_buf_ops },
+	{ TYP_AGFL, "agfl", handle_struct, agfl_crc_hfld, &xfs_agfl_buf_ops },
+	{ TYP_AGI, "agi", handle_struct, agi_hfld, &xfs_agfl_buf_ops },
+	{ TYP_ATTR, "attr3", handle_struct, attr3_hfld, NULL },
+	{ TYP_BMAPBTA, "bmapbta", handle_struct, bmapbta_crc_hfld,
+		&xfs_bmbt_buf_ops },
+	{ TYP_BMAPBTD, "bmapbtd", handle_struct, bmapbtd_crc_hfld,
+		&xfs_bmbt_buf_ops },
+	{ TYP_BNOBT, "bnobt", handle_struct, bnobt_crc_hfld,
+		&xfs_allocbt_buf_ops },
+	{ TYP_CNTBT, "cntbt", handle_struct, cntbt_crc_hfld,
+		&xfs_allocbt_buf_ops },
+	{ TYP_DATA, "data", handle_block, NULL, NULL },
+	{ TYP_DIR2, "dir3", handle_struct, dir3_hfld, NULL },
+	{ TYP_DQBLK, "dqblk", handle_struct, dqblk_hfld,
+		&xfs_dquot_buf_ops },
+	{ TYP_INOBT, "inobt", handle_struct, inobt_crc_hfld,
+		&xfs_inobt_buf_ops },
+	{ TYP_INODATA, "inodata", NULL, NULL, NULL },
+	{ TYP_INODE, "inode", handle_struct, inode_crc_hfld,
+		&xfs_inode_buf_ops },
+	{ TYP_LOG, "log", NULL, NULL, NULL },
+	{ TYP_RTBITMAP, "rtbitmap", NULL, NULL, NULL },
+	{ TYP_RTSUMMARY, "rtsummary", NULL, NULL, NULL },
+	{ TYP_SB, "sb", handle_struct, sb_hfld, &xfs_sb_buf_ops },
+	{ TYP_SYMLINK, "symlink", handle_struct, symlink_crc_hfld,
+		&xfs_symlink_buf_ops },
+	{ TYP_TEXT, "text", handle_text, NULL, NULL },
 	{ TYP_NONE, NULL }
 };
 
diff --git a/db/type.h b/db/type.h
index c41aca4d..3bb26f1 100644
--- a/db/type.h
+++ b/db/type.h
@@ -42,6 +42,7 @@ typedef struct typ
 	char			*name;
 	pfunc_t			pfunc;
 	const struct field	*fields;
+	const struct xfs_buf_ops *bops;
 } typ_t;
 extern const typ_t	*typtab, *cur_typ;
 
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [RFC PATCH 07/12] db: indicate if the CRC on a buffer is correct or not
  2013-09-05 12:13 [RFC PATCH 00/12] xfs_db: write support for v5 filesystems Dave Chinner
                   ` (5 preceding siblings ...)
  2013-09-05 12:13 ` [RFC PATCH 06/12] db: introduce verifier support into set_cur Dave Chinner
@ 2013-09-05 12:13 ` Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 08/12] db: verify and calculate inode CRCs Dave Chinner
                   ` (4 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2013-09-05 12:13 UTC (permalink / raw)
  To: xfs

From: Dave Chinner <dchinner@redhat.com>

When dumping metadata that has a CRC in it, output not only the CRC
but text to tell us whether the value is correct or not. Hence we
can see at a glance if there's something wrong or not.

Do this by peeking at the buffer attached to the current IO
contexted. If there was a CRC error, then it will be marked with a
EFSCORRUPTED error. Use this to determine what to output.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 db/agf.c     |  2 +-
 db/agfl.c    |  2 +-
 db/agi.c     |  2 +-
 db/btblock.c | 10 +++++-----
 db/dir2.c    |  4 ++--
 db/dquot.c   |  2 +-
 db/field.c   |  5 +++++
 db/field.h   |  4 ++++
 db/fprint.c  | 39 +++++++++++++++++++++++++++++++++++++++
 db/fprint.h  |  2 ++
 db/inode.c   |  2 +-
 db/io.h      |  6 ++++++
 db/sb.c      |  2 +-
 db/symlink.c |  2 +-
 14 files changed, 70 insertions(+), 14 deletions(-)

diff --git a/db/agf.c b/db/agf.c
index 389cb43..d9a07ca 100644
--- a/db/agf.c
+++ b/db/agf.c
@@ -71,7 +71,7 @@ const field_t	agf_flds[] = {
 	{ "btreeblks", FLDT_UINT32D, OI(OFF(btreeblks)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(uuid)), C1, 0, TYP_NONE },
 	{ "lsn", FLDT_UINT64X, OI(OFF(lsn)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE },
 	{ NULL }
 };
 
diff --git a/db/agfl.c b/db/agfl.c
index e2340e6..b29538f 100644
--- a/db/agfl.c
+++ b/db/agfl.c
@@ -58,7 +58,7 @@ const field_t	agfl_crc_flds[] = {
 	{ "seqno", FLDT_AGNUMBER, OI(OFF(seqno)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(uuid)), C1, 0, TYP_NONE },
 	{ "lsn", FLDT_UINT64X, OI(OFF(lsn)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE },
 	{ "bno", FLDT_AGBLOCKNZ, OI(OFF(bno)), agfl_bno_size,
 	  FLD_ARRAY|FLD_COUNT, TYP_DATA },
 	{ NULL }
diff --git a/db/agi.c b/db/agi.c
index 6b2e889..398bdbb 100644
--- a/db/agi.c
+++ b/db/agi.c
@@ -56,7 +56,7 @@ const field_t	agi_flds[] = {
 	  CI(XFS_AGI_UNLINKED_BUCKETS), FLD_ARRAY, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(uuid)), C1, 0, TYP_NONE },
 	{ "lsn", FLDT_UINT64X, OI(OFF(lsn)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE },
 	{ NULL }
 };
 
diff --git a/db/btblock.c b/db/btblock.c
index 34188db..1ea0cff 100644
--- a/db/btblock.c
+++ b/db/btblock.c
@@ -295,7 +295,7 @@ const field_t	bmapbta_crc_flds[] = {
 	{ "lsn", FLDT_UINT64X, OI(OFF(u.l.bb_lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(u.l.bb_uuid)), C1, 0, TYP_NONE },
 	{ "owner", FLDT_INO, OI(OFF(u.l.bb_owner)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(u.l.bb_crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(u.l.bb_crc)), C1, 0, TYP_NONE },
 	{ "recs", FLDT_BMAPBTAREC, btblock_rec_offset, btblock_rec_count,
 	  FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE },
 	{ "keys", FLDT_BMAPBTAKEY, btblock_key_offset, btblock_key_count,
@@ -314,7 +314,7 @@ const field_t	bmapbtd_crc_flds[] = {
 	{ "lsn", FLDT_UINT64X, OI(OFF(u.l.bb_lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(u.l.bb_uuid)), C1, 0, TYP_NONE },
 	{ "owner", FLDT_INO, OI(OFF(u.l.bb_owner)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(u.l.bb_crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(u.l.bb_crc)), C1, 0, TYP_NONE },
 	{ "recs", FLDT_BMAPBTDREC, btblock_rec_offset, btblock_rec_count,
 	  FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE },
 	{ "keys", FLDT_BMAPBTDKEY, btblock_key_offset, btblock_key_count,
@@ -405,7 +405,7 @@ const field_t	inobt_crc_flds[] = {
 	{ "lsn", FLDT_UINT64X, OI(OFF(u.s.bb_lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(u.s.bb_uuid)), C1, 0, TYP_NONE },
 	{ "owner", FLDT_AGNUMBER, OI(OFF(u.s.bb_owner)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE },
 	{ "recs", FLDT_INOBTREC, btblock_rec_offset, btblock_rec_count,
 	  FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE },
 	{ "keys", FLDT_INOBTKEY, btblock_key_offset, btblock_key_count,
@@ -471,7 +471,7 @@ const field_t	bnobt_crc_flds[] = {
 	{ "lsn", FLDT_UINT64X, OI(OFF(u.s.bb_lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(u.s.bb_uuid)), C1, 0, TYP_NONE },
 	{ "owner", FLDT_AGNUMBER, OI(OFF(u.s.bb_owner)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE },
 	{ "recs", FLDT_BNOBTREC, btblock_rec_offset, btblock_rec_count,
 	  FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE },
 	{ "keys", FLDT_BNOBTKEY, btblock_key_offset, btblock_key_count,
@@ -533,7 +533,7 @@ const field_t	cntbt_crc_flds[] = {
 	{ "lsn", FLDT_UINT64X, OI(OFF(u.s.bb_lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(u.s.bb_uuid)), C1, 0, TYP_NONE },
 	{ "owner", FLDT_AGNUMBER, OI(OFF(u.s.bb_owner)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(u.s.bb_crc)), C1, 0, TYP_NONE },
 	{ "recs", FLDT_CNTBTREC, btblock_rec_offset, btblock_rec_count,
 	  FLD_ARRAY|FLD_ABASE1|FLD_COUNT|FLD_OFFSET, TYP_NONE },
 	{ "keys", FLDT_CNTBTKEY, btblock_key_offset, btblock_key_count,
diff --git a/db/dir2.c b/db/dir2.c
index 8b08d48..2ec64e0 100644
--- a/db/dir2.c
+++ b/db/dir2.c
@@ -922,7 +922,7 @@ const field_t	dir3_data_union_flds[] = {
 #define	DBH3OFF(f)	bitize(offsetof(struct xfs_dir3_blk_hdr, f))
 const field_t	dir3_blkhdr_flds[] = {
 	{ "magic", FLDT_UINT32X, OI(DBH3OFF(magic)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(DBH3OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(DBH3OFF(crc)), C1, 0, TYP_NONE },
 	{ "bno", FLDT_DFSBNO, OI(DBH3OFF(blkno)), C1, 0, TYP_BMAPBTD },
 	{ "lsn", FLDT_UINT64X, OI(DBH3OFF(lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(DBH3OFF(uuid)), C1, 0, TYP_NONE },
@@ -959,7 +959,7 @@ const field_t	dir3_free_hdr_flds[] = {
 #define	DB3OFF(f)	bitize(offsetof(struct xfs_da3_blkinfo, f))
 const field_t	da3_blkinfo_flds[] = {
 	{ "hdr", FLDT_DA_BLKINFO, OI(DB3OFF(hdr)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(DB3OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(DB3OFF(crc)), C1, 0, TYP_NONE },
 	{ "bno", FLDT_DFSBNO, OI(DB3OFF(blkno)), C1, 0, TYP_BMAPBTD },
 	{ "lsn", FLDT_UINT64X, OI(DB3OFF(lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(DB3OFF(uuid)), C1, 0, TYP_NONE },
diff --git a/db/dquot.c b/db/dquot.c
index 6927956..2f7d463 100644
--- a/db/dquot.c
+++ b/db/dquot.c
@@ -48,7 +48,7 @@ const field_t	dqblk_flds[] = {
 	{ "diskdq", FLDT_DISK_DQUOT, OI(DDOFF(diskdq)), C1, 0, TYP_NONE },
 	{ "fill", FLDT_CHARS, OI(DDOFF(fill)), CI(DDSZC(fill)), FLD_SKIPALL,
 	  TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(DDOFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(DDOFF(crc)), C1, 0, TYP_NONE },
 	{ "lsn", FLDT_UINT64X, OI(DDOFF(lsn)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(DDOFF(uuid)), C1, 0, TYP_NONE },
 	{ NULL }
diff --git a/db/field.c b/db/field.c
index c6d7404..4582097 100644
--- a/db/field.c
+++ b/db/field.c
@@ -163,6 +163,11 @@ const ftattr_t	ftattrtab[] = {
 	  0, fa_agblock, NULL },
 	{ FLDT_CNTBTREC, "cntbtrec", fp_sarray, (char *)cntbt_rec_flds,
 	  SI(bitsz(xfs_alloc_rec_t)), 0, NULL, cntbt_rec_flds },
+
+/* CRC field */
+	{ FLDT_CRC, "crc", fp_crc, "%#x (%s)", SI(bitsz(__uint32_t)),
+	  0, NULL, NULL },
+
 	{ FLDT_DEV, "dev", fp_num, "%#x", SI(bitsz(xfs_dev_t)), 0, NULL, NULL },
 	{ FLDT_DFILOFFA, "dfiloffa", fp_num, "%llu", SI(bitsz(xfs_dfiloff_t)),
 	  0, fa_dfiloffa, NULL },
diff --git a/db/field.h b/db/field.h
index aecdf9f..6343c9a 100644
--- a/db/field.h
+++ b/db/field.h
@@ -80,6 +80,10 @@ typedef enum fldt	{
 	FLDT_CNTBTKEY,
 	FLDT_CNTBTPTR,
 	FLDT_CNTBTREC,
+
+	/* CRC field type */
+	FLDT_CRC,
+
 	FLDT_DEV,
 	FLDT_DFILOFFA,
 	FLDT_DFILOFFD,
diff --git a/db/fprint.c b/db/fprint.c
index 1d2f29c..435d984 100644
--- a/db/fprint.c
+++ b/db/fprint.c
@@ -30,6 +30,7 @@
 #include "output.h"
 #include "sig.h"
 #include "malloc.h"
+#include "io.h"
 
 int
 fp_charns(
@@ -184,3 +185,41 @@ fp_uuid(
 	}
 	return 1;
 }
+
+/*
+ * CRC is correct is the current buffer it is being pulled out
+ * of is not marked with a EFSCORRUPTED error.
+ */
+int
+fp_crc(
+	void	*obj,
+	int	bit,
+	int	count,
+	char	*fmtstr,
+	int	size,
+	int	arg,
+	int	base,
+	int	array)
+{
+	int		bitpos;
+	int		i;
+	__int64_t	val;
+	char		*ok;
+
+	ok = iocur_crc_valid() ? "correct" : "bad";
+
+	for (i = 0, bitpos = bit;
+	     i < count && !seenint();
+	     i++, bitpos += size) {
+		if (array)
+			dbprintf("%d:", i + base);
+		val = getbitval(obj, bitpos, size, BVUNSIGNED);
+		if (size > 32)
+			dbprintf(fmtstr, val, ok);
+		else
+			dbprintf(fmtstr, (__int32_t)val, ok);
+		if (i < count - 1)
+			dbprintf(" ");
+	}
+	return 1;
+}
diff --git a/db/fprint.h b/db/fprint.h
index b032dbd..6a6d77e 100644
--- a/db/fprint.h
+++ b/db/fprint.h
@@ -29,3 +29,5 @@ extern int	fp_time(void *obj, int bit, int count, char *fmtstr, int size,
 			int arg, int base, int array);
 extern int	fp_uuid(void *obj, int bit, int count, char *fmtstr, int size,
 			int arg, int base, int array);
+extern int	fp_crc(void *obj, int bit, int count, char *fmtstr, int size,
+		       int arg, int base, int array);
diff --git a/db/inode.c b/db/inode.c
index 634dc30..ec533ee 100644
--- a/db/inode.c
+++ b/db/inode.c
@@ -168,7 +168,7 @@ const field_t	inode_core_flds[] = {
 };
 
 const field_t	inode_v3_flds[] = {
-	{ "crc", FLDT_UINT32X, OI(COFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(COFF(crc)), C1, 0, TYP_NONE },
 	{ "change_count", FLDT_UINT64D, OI(COFF(changecount)), C1, 0, TYP_NONE },
 	{ "lsn", FLDT_UINT64X, OI(COFF(lsn)), C1, 0, TYP_NONE },
 	{ "flags2", FLDT_UINT64X, OI(COFF(flags2)), C1, 0, TYP_NONE },
diff --git a/db/io.h b/db/io.h
index 2c47ccc..d647284 100644
--- a/db/io.h
+++ b/db/io.h
@@ -58,3 +58,9 @@ extern void     write_cur(void);
 extern void	set_cur(const struct typ *t, __int64_t d, int c, int ring_add,
 			bbmap_t *bbmap);
 extern void     ring_add(void);
+
+static inline bool
+iocur_crc_valid()
+{
+	return (iocur_top->bp && iocur_top->bp->b_error != EFSCORRUPTED);
+}
diff --git a/db/sb.c b/db/sb.c
index d178f58..b36abf8 100644
--- a/db/sb.c
+++ b/db/sb.c
@@ -118,7 +118,7 @@ const field_t	sb_flds[] = {
 		C1, 0, TYP_NONE },
 	{ "features_log_incompat", FLDT_UINT32X, OI(OFF(features_log_incompat)),
 		C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE },
 	{ "pquotino", FLDT_INO, OI(OFF(pquotino)), C1, 0, TYP_INODE },
 	{ "lsn", FLDT_UINT64X, OI(OFF(lsn)), C1, 0, TYP_NONE },
 	{ NULL }
diff --git a/db/symlink.c b/db/symlink.c
index 9f3d0b9..a4f420f 100644
--- a/db/symlink.c
+++ b/db/symlink.c
@@ -69,7 +69,7 @@ const struct field	symlink_crc_flds[] = {
 	{ "magic", FLDT_UINT32X, OI(OFF(magic)), C1, 0, TYP_NONE },
 	{ "offset", FLDT_UINT32D, OI(OFF(offset)), C1, 0, TYP_NONE },
 	{ "bytes", FLDT_UINT32D, OI(OFF(bytes)), C1, 0, TYP_NONE },
-	{ "crc", FLDT_UINT32X, OI(OFF(crc)), C1, 0, TYP_NONE },
+	{ "crc", FLDT_CRC, OI(OFF(crc)), C1, 0, TYP_NONE },
 	{ "uuid", FLDT_UUID, OI(OFF(uuid)), C1, 0, TYP_NONE },
 	{ "owner", FLDT_INO, OI(OFF(owner)), C1, 0, TYP_NONE },
 	{ "bno", FLDT_DFSBNO, OI(OFF(blkno)), C1, 0, TYP_BMAPBTD },
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [RFC PATCH 08/12] db: verify and calculate inode CRCs
  2013-09-05 12:13 [RFC PATCH 00/12] xfs_db: write support for v5 filesystems Dave Chinner
                   ` (6 preceding siblings ...)
  2013-09-05 12:13 ` [RFC PATCH 07/12] db: indicate if the CRC on a buffer is correct or not Dave Chinner
@ 2013-09-05 12:13 ` Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 09/12] db: verify and calculate dquot CRCs Dave Chinner
                   ` (3 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2013-09-05 12:13 UTC (permalink / raw)
  To: xfs

From: Dave Chinner <dchinner@redhat.com>

When we set the current Io cursor to point at an inode, verify that
the inode CRC is intact. And prior to writing such an IO cursor,
calculate the inode CRC.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 db/inode.c             | 2 ++
 db/io.c                | 4 ++++
 db/io.h                | 6 +++++-
 include/libxfs.h       | 4 ++++
 libxfs/xfs_inode_buf.c | 8 ++++----
 5 files changed, 19 insertions(+), 5 deletions(-)

diff --git a/db/inode.c b/db/inode.c
index ec533ee..4090855 100644
--- a/db/inode.c
+++ b/db/inode.c
@@ -655,6 +655,8 @@ set_cur_inode(
 		blkbb, DB_RING_IGN, NULL);
 	off_cur(offset << mp->m_sb.sb_inodelog, mp->m_sb.sb_inodesize);
 	dip = iocur_top->data;
+	iocur_top->ino_crc_ok = libxfs_dinode_verify(mp, ino, dip);
+	iocur_top->ino_buf = 1;
 	iocur_top->ino = ino;
 	iocur_top->mode = be16_to_cpu(dip->di_mode);
 	if ((iocur_top->mode & S_IFMT) == S_IFDIR)
diff --git a/db/io.c b/db/io.c
index d6e1d28..5f2e492 100644
--- a/db/io.c
+++ b/db/io.c
@@ -462,6 +462,9 @@ write_cur(void)
 		return;
 	}
 
+	if (iocur_top->ino_buf)
+		libxfs_dinode_calc_crc(mp, iocur_top->data);
+
 	if (iocur_top->bbmap)
 		write_cur_bbs();
 	else
@@ -528,6 +531,7 @@ set_cur(
 	iocur_top->ino = ino;
 	iocur_top->dirino = dirino;
 	iocur_top->mode = mode;
+	iocur_top->ino_buf = 0;
 
 	/* store location in ring */
 	if (ring_flag)
diff --git a/db/io.h b/db/io.h
index d647284..1f8270d 100644
--- a/db/io.h
+++ b/db/io.h
@@ -38,6 +38,8 @@ typedef struct iocur {
 	const struct typ	*typ;	/* type of "data" */
 	bbmap_t			*bbmap;	/* map daddr if fragmented */
 	struct xfs_buf		*bp;	/* underlying buffer */
+	int			ino_crc_ok:1;
+	int			ino_buf:1;
 } iocur_t;
 
 #define DB_RING_ADD 1                   /* add to ring on set_cur */
@@ -62,5 +64,7 @@ extern void     ring_add(void);
 static inline bool
 iocur_crc_valid()
 {
-	return (iocur_top->bp && iocur_top->bp->b_error != EFSCORRUPTED);
+	return (iocur_top->bp &&
+		iocur_top->bp->b_error != EFSCORRUPTED &&
+		(!iocur_top->ino_buf || iocur_top->ino_crc_ok));
 }
diff --git a/include/libxfs.h b/include/libxfs.h
index 9429f61..90f9a75 100644
--- a/include/libxfs.h
+++ b/include/libxfs.h
@@ -754,6 +754,10 @@ void	xfs_dinode_from_disk(struct xfs_icdinode *,
 #define libxfs_idata_realloc		xfs_idata_realloc
 #define libxfs_idestroy_fork		xfs_idestroy_fork
 
+#define libxfs_dinode_verify		xfs_dinode_verify
+bool xfs_dinode_verify(struct xfs_mount *mp, xfs_ino_t ino,
+		       struct xfs_dinode *dip);
+
 /* xfs_sb.h */
 #define libxfs_mod_sb			xfs_mod_sb
 #define libxfs_sb_from_disk		xfs_sb_from_disk
diff --git a/libxfs/xfs_inode_buf.c b/libxfs/xfs_inode_buf.c
index 6205318..54ad320 100644
--- a/libxfs/xfs_inode_buf.c
+++ b/libxfs/xfs_inode_buf.c
@@ -246,10 +246,10 @@ xfs_dinode_to_disk(
 	}
 }
 
-static bool
+bool
 xfs_dinode_verify(
 	struct xfs_mount	*mp,
-	struct xfs_inode	*ip,
+	xfs_ino_t		ino,
 	struct xfs_dinode	*dip)
 {
 	if (dip->di_magic != cpu_to_be16(XFS_DINODE_MAGIC))
@@ -264,7 +264,7 @@ xfs_dinode_verify(
 	if (!xfs_verify_cksum((char *)dip, mp->m_sb.sb_inodesize,
 			      offsetof(struct xfs_dinode, di_crc)))
 		return false;
-	if (be64_to_cpu(dip->di_ino) != ip->i_ino)
+	if (be64_to_cpu(dip->di_ino) != ino)
 		return false;
 	if (!uuid_equal(&dip->di_uuid, &mp->m_sb.sb_uuid))
 		return false;
@@ -315,7 +315,7 @@ xfs_iread(
 		return error;
 
 	/* even unallocated inodes are verified */
-	if (!xfs_dinode_verify(mp, ip, dip)) {
+	if (!xfs_dinode_verify(mp, ip->i_ino, dip)) {
 		xfs_alert(mp, "%s: validation failed for inode %lld failed",
 				__func__, ip->i_ino);
 
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [RFC PATCH 09/12] db: verify and calculate dquot CRCs
  2013-09-05 12:13 [RFC PATCH 00/12] xfs_db: write support for v5 filesystems Dave Chinner
                   ` (7 preceding siblings ...)
  2013-09-05 12:13 ` [RFC PATCH 08/12] db: verify and calculate inode CRCs Dave Chinner
@ 2013-09-05 12:13 ` Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 10/12] db: add a special directory buffer verifier Dave Chinner
                   ` (2 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2013-09-05 12:13 UTC (permalink / raw)
  To: xfs

When we set the current Io cursor to point at a dquot block, verify
that the dquot CRC is intact. And prior to writing such an IO
cursor, calculate the dquot CRC.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 db/io.c | 5 ++++-
 db/io.h | 1 +
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/db/io.c b/db/io.c
index 5f2e492..04f1879 100644
--- a/db/io.c
+++ b/db/io.c
@@ -464,7 +464,9 @@ write_cur(void)
 
 	if (iocur_top->ino_buf)
 		libxfs_dinode_calc_crc(mp, iocur_top->data);
-
+	if (iocur_top->dquot_buf)
+		xfs_update_cksum(iocur_top->data, sizeof(struct xfs_dqblk),
+				 XFS_DQUOT_CRC_OFF);
 	if (iocur_top->bbmap)
 		write_cur_bbs();
 	else
@@ -532,6 +534,7 @@ set_cur(
 	iocur_top->dirino = dirino;
 	iocur_top->mode = mode;
 	iocur_top->ino_buf = 0;
+	iocur_top->dquot_buf = 0;
 
 	/* store location in ring */
 	if (ring_flag)
diff --git a/db/io.h b/db/io.h
index 1f8270d..4f24c83 100644
--- a/db/io.h
+++ b/db/io.h
@@ -40,6 +40,7 @@ typedef struct iocur {
 	struct xfs_buf		*bp;	/* underlying buffer */
 	int			ino_crc_ok:1;
 	int			ino_buf:1;
+	int			dquot_buf:1;
 } iocur_t;
 
 #define DB_RING_ADD 1                   /* add to ring on set_cur */
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [RFC PATCH 10/12] db: add a special directory buffer verifier
  2013-09-05 12:13 [RFC PATCH 00/12] xfs_db: write support for v5 filesystems Dave Chinner
                   ` (8 preceding siblings ...)
  2013-09-05 12:13 ` [RFC PATCH 09/12] db: verify and calculate dquot CRCs Dave Chinner
@ 2013-09-05 12:13 ` Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 11/12] db: add a special attribute " Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 12/12] db: re-enable write support for v5 filesystems Dave Chinner
  11 siblings, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2013-09-05 12:13 UTC (permalink / raw)
  To: xfs

From: Dave Chinner <dchinner@redhat.com>

Because we only have a single directory type that is used for all
the different buffer types, we need to provide a special verifier
for the read code. That verifier needs to know all the directory
types and when it find one it knows about, switch to the correct
verifier and call it.

We already do this for certain readahead cases in the directory
code, so there is precedence for this. If we don't find a magic
number we recognise, the verifier fails...

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 db/dir2.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 db/dir2.h |  2 ++
 db/type.c |  3 ++-
 3 files changed, 65 insertions(+), 1 deletion(-)

diff --git a/db/dir2.c b/db/dir2.c
index 2ec64e0..5a10955 100644
--- a/db/dir2.c
+++ b/db/dir2.c
@@ -24,6 +24,7 @@
 #include "field.h"
 #include "dir2.h"
 #include "init.h"
+#include "output.h"
 
 static int	dir2_block_hdr_count(void *obj, int startoff);
 static int	dir2_block_leaf_count(void *obj, int startoff);
@@ -975,3 +976,63 @@ const field_t	da3_node_hdr_flds[] = {
 	{ "pad", FLDT_UINT32D, OI(H3OFF(__pad32)), C1, 0, TYP_NONE },
 	{ NULL }
 };
+
+/*
+ * Special read verifier for directory buffers. detect the magic number
+ * appropriately and set the correct verifier and call it.
+ */
+static void
+xfs_dir3_db_read_verify(
+	struct xfs_buf		*bp)
+{
+	__be32			magic32;
+	__be16			magic16;
+
+	magic32 = *(__be32 *)bp->b_addr;
+	magic16 = ((struct xfs_da_blkinfo *)bp->b_addr)->magic;
+
+	switch (magic32) {
+	case cpu_to_be32(XFS_DIR3_BLOCK_MAGIC):
+		bp->b_ops = &xfs_dir3_block_buf_ops;
+		goto verify;
+	case cpu_to_be32(XFS_DIR3_DATA_MAGIC):
+		bp->b_ops = &xfs_dir3_data_buf_ops;
+		goto verify;
+	case cpu_to_be32(XFS_DIR3_FREE_MAGIC):
+		bp->b_ops = &xfs_dir3_free_buf_ops;
+		goto verify;
+	default:
+		break;
+	}
+
+	switch (magic16) {
+	case cpu_to_be16(XFS_DIR3_LEAF1_MAGIC):
+		bp->b_ops = &xfs_dir3_leaf1_buf_ops;
+		break;
+	case cpu_to_be16(XFS_DIR3_LEAFN_MAGIC):
+		bp->b_ops = &xfs_dir3_leafn_buf_ops;
+		break;
+	case cpu_to_be16(XFS_DA3_NODE_MAGIC):
+		bp->b_ops = &xfs_da3_node_buf_ops;
+		break;
+	default:
+		dbprintf(_("Unknown directory buffer type!\n"));
+		xfs_buf_ioerror(bp, EFSCORRUPTED);
+		return;
+	}
+verify:
+	bp->b_ops->verify_read(bp);
+}
+
+static void
+xfs_dir3_db_write_verify(
+	struct xfs_buf		*bp)
+{
+	dbprintf(_("Writing unknown directory buffer type!\n"));
+	xfs_buf_ioerror(bp, EFSCORRUPTED);
+}
+
+const struct xfs_buf_ops xfs_dir3_db_buf_ops = {
+	.verify_read = xfs_dir3_db_read_verify,
+	.verify_write = xfs_dir3_db_write_verify,
+};
diff --git a/db/dir2.h b/db/dir2.h
index b3651d5..5054493 100644
--- a/db/dir2.h
+++ b/db/dir2.h
@@ -60,3 +60,5 @@ static inline xfs_dir2_inou_t *xfs_dir2_sf_inumberp(xfs_dir2_sf_entry_t *sfep)
 
 extern int	dir2_data_union_size(void *obj, int startoff, int idx);
 extern int	dir2_size(void *obj, int startoff, int idx);
+
+extern const struct xfs_buf_ops xfs_dir3_db_buf_ops;
diff --git a/db/type.c b/db/type.c
index b3f3d87..2c3431e 100644
--- a/db/type.c
+++ b/db/type.c
@@ -87,7 +87,8 @@ static const typ_t	__typtab_crc[] = {
 	{ TYP_CNTBT, "cntbt", handle_struct, cntbt_crc_hfld,
 		&xfs_allocbt_buf_ops },
 	{ TYP_DATA, "data", handle_block, NULL, NULL },
-	{ TYP_DIR2, "dir3", handle_struct, dir3_hfld, NULL },
+	{ TYP_DIR2, "dir3", handle_struct, dir3_hfld,
+		&xfs_dir3_db_buf_ops },
 	{ TYP_DQBLK, "dqblk", handle_struct, dqblk_hfld,
 		&xfs_dquot_buf_ops },
 	{ TYP_INOBT, "inobt", handle_struct, inobt_crc_hfld,
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [RFC PATCH 11/12] db: add a special attribute buffer verifier
  2013-09-05 12:13 [RFC PATCH 00/12] xfs_db: write support for v5 filesystems Dave Chinner
                   ` (9 preceding siblings ...)
  2013-09-05 12:13 ` [RFC PATCH 10/12] db: add a special directory buffer verifier Dave Chinner
@ 2013-09-05 12:13 ` Dave Chinner
  2013-09-05 12:13 ` [RFC PATCH 12/12] db: re-enable write support for v5 filesystems Dave Chinner
  11 siblings, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2013-09-05 12:13 UTC (permalink / raw)
  To: xfs

From: Dave Chinner <dchinner@redhat.com>

Because we only have a single attribute type that is used for all
the attribute buffer types, we need to provide a special verifier
for the read code. That verifier needs to know all the attribute
types and when it find one it knows about, switch to the correct
verifier and call it.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 db/attr.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 db/attr.h |  2 ++
 db/type.c |  3 ++-
 3 files changed, 55 insertions(+), 1 deletion(-)

diff --git a/db/attr.c b/db/attr.c
index cd95a0a..359af7b 100644
--- a/db/attr.c
+++ b/db/attr.c
@@ -25,6 +25,7 @@
 #include "attr.h"
 #include "io.h"
 #include "init.h"
+#include "output.h"
 
 static int	attr_leaf_entries_count(void *obj, int startoff);
 static int	attr_leaf_hdr_count(void *obj, int startoff);
@@ -522,3 +523,53 @@ const field_t	attr3_leaf_hdr_flds[] = {
 	{ NULL }
 };
 
+/*
+ * Special read verifier for attribute buffers. detect the magic number
+ * appropriately and set the correct verifier and call it.
+ */
+static void
+xfs_attr3_db_read_verify(
+	struct xfs_buf		*bp)
+{
+	__be32			magic32;
+	__be16			magic16;
+
+	magic32 = *(__be32 *)bp->b_addr;
+	magic16 = ((struct xfs_da_blkinfo *)bp->b_addr)->magic;
+
+	switch (magic16) {
+	case cpu_to_be16(XFS_ATTR3_LEAF_MAGIC):
+		bp->b_ops = &xfs_attr3_leaf_buf_ops;
+		goto verify;
+	case cpu_to_be16(XFS_DA3_NODE_MAGIC):
+		bp->b_ops = &xfs_da3_node_buf_ops;
+		goto verify;
+	default:
+		break;
+	}
+
+	switch (magic32) {
+	case cpu_to_be32(XFS_ATTR3_RMT_MAGIC):
+		bp->b_ops = &xfs_attr3_rmt_buf_ops;
+		break;
+	default:
+		dbprintf(_("Unknown attribute buffer type!\n"));
+		xfs_buf_ioerror(bp, EFSCORRUPTED);
+		return;
+	}
+verify:
+	bp->b_ops->verify_read(bp);
+}
+
+static void
+xfs_attr3_db_write_verify(
+	struct xfs_buf		*bp)
+{
+	dbprintf(_("Writing unknown attribute buffer type!\n"));
+	xfs_buf_ioerror(bp, EFSCORRUPTED);
+}
+
+const struct xfs_buf_ops xfs_attr3_db_buf_ops = {
+	.verify_read = xfs_attr3_db_read_verify,
+	.verify_write = xfs_attr3_db_write_verify,
+};
diff --git a/db/attr.h b/db/attr.h
index 3065372..bc3431f 100644
--- a/db/attr.h
+++ b/db/attr.h
@@ -33,3 +33,5 @@ extern const field_t	attr3_node_hdr_flds[];
 
 extern int	attr_leaf_name_size(void *obj, int startoff, int idx);
 extern int	attr_size(void *obj, int startoff, int idx);
+
+extern const struct xfs_buf_ops xfs_attr3_db_buf_ops;
diff --git a/db/type.c b/db/type.c
index 2c3431e..04d0d56 100644
--- a/db/type.c
+++ b/db/type.c
@@ -77,7 +77,8 @@ static const typ_t	__typtab_crc[] = {
 	{ TYP_AGF, "agf", handle_struct, agf_hfld, &xfs_agf_buf_ops },
 	{ TYP_AGFL, "agfl", handle_struct, agfl_crc_hfld, &xfs_agfl_buf_ops },
 	{ TYP_AGI, "agi", handle_struct, agi_hfld, &xfs_agfl_buf_ops },
-	{ TYP_ATTR, "attr3", handle_struct, attr3_hfld, NULL },
+	{ TYP_ATTR, "attr3", handle_struct, attr3_hfld,
+		&xfs_attr3_db_buf_ops },
 	{ TYP_BMAPBTA, "bmapbta", handle_struct, bmapbta_crc_hfld,
 		&xfs_bmbt_buf_ops },
 	{ TYP_BMAPBTD, "bmapbtd", handle_struct, bmapbtd_crc_hfld,
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply related	[flat|nested] 13+ messages in thread
* [RFC PATCH 12/12] db: re-enable write support for v5 filesystems.
  2013-09-05 12:13 [RFC PATCH 00/12] xfs_db: write support for v5 filesystems Dave Chinner
                   ` (10 preceding siblings ...)
  2013-09-05 12:13 ` [RFC PATCH 11/12] db: add a special attribute " Dave Chinner
@ 2013-09-05 12:13 ` Dave Chinner
  11 siblings, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2013-09-05 12:13 UTC (permalink / raw)
  To: xfs

From: Dave Chinner <dchinner@redhat.com>

As we can now verify and recalculate CRCs on IO, we can modify the
on-disk structures without corrupting the filesyste, This makes it
safe to turn write support on for v5 filesystems for the first time.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 db/init.c | 15 ---------------
 1 file changed, 15 deletions(-)

diff --git a/db/init.c b/db/init.c
index b1cddca..a9b357b 100644
--- a/db/init.c
+++ b/db/init.c
@@ -143,21 +143,6 @@ init(
 			exit(EXIT_FAILURE);
 	}
 
-	/*
-	 * Don't allow modifications to CRC enabled filesystems until we support
-	 * CRC recalculation in the IO path. Unless, of course, the user is in
-	 * the process of hitting us with a big hammer.
-	 */
-	if (XFS_SB_VERSION_NUM(sbp) >= XFS_SB_VERSION_5 &&
-	    !(x.isreadonly & LIBXFS_ISREADONLY)) {
-		fprintf(stderr, 
-	_("%s: modifications to %s are not supported in thi version.\n"
-	"Use \"-r\" to run %s in read-only mode on this filesystem .\n"),
-			progname, fsdevice, progname);
-		if (!force)
-			exit(EXIT_FAILURE);
-	}
-
 	mp = libxfs_mount(&xmount, sbp, x.ddev, x.logdev, x.rtdev,
 				LIBXFS_MOUNT_ROOTINOS | LIBXFS_MOUNT_DEBUGGER);
 	if (!mp) {
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply related	[flat|nested] 13+ messages in thread
end of thread, other threads:[~2013-09-05 12:13 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-09-05 12:13 [RFC PATCH 00/12] xfs_db: write support for v5 filesystems Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 01/12] db: separate out straight buffer IO from map based IO Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 02/12] db: rewrite bbmap to use xfs_buf_map Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 03/12] db: rewrite IO engine to use libxfs Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 04/12] xfs: split dquot buffer operations out Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 05/12] xfs: move buf_ops externs to a xfs_format.h Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 06/12] db: introduce verifier support into set_cur Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 07/12] db: indicate if the CRC on a buffer is correct or not Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 08/12] db: verify and calculate inode CRCs Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 09/12] db: verify and calculate dquot CRCs Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 10/12] db: add a special directory buffer verifier Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 11/12] db: add a special attribute " Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 12/12] db: re-enable write support for v5 filesystems Dave Chinner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox