editable udf metadata

editable udf metadata

[Pat LaVarre]

> > Do you mean to say you see the metadata ...

I know I haven't yet found a convenient printer of metadata for udf
files.

I mean like a tool to tell me the file name and byte size and last
modified time etc. appears in block xyz, the first bytes of data appears
at offset xyz in block xyz, the next block of data appears in block xyz,
and so on til eof.

I of course dream of an editor that would let me browse and edit the
blocks of the disk, having presented those blocks like gdb would present
the udf.ko struct's if udf.ko were a user app.

Pat LaVarre

Re: editable udf metadata

[Pat LaVarre]

[-- Attachment #1: Type: text/plain, Size: 1518 bytes --]

> > > Do you mean to say you see the metadata ...
> > 
> > I know I haven't yet found
> > a convenient printer of metadata for udf files.

Kindly offline I received the ...philips.com/udf/ link.

I wonder if that means people somewhere know how to extract just the
metadata of a single file from the udfct_1_0r2 tool.

The web trail I know to the udfct_1_0r2 tool is:

http://www.google.com/search?q=philips+udf+verifier
http://www.google.com/search?q=philips+udf+verifier&btnI=1
http://www.extra.research.philips.com/udf/
http://www.extra.research.philips.com/udf/download.html

I see udfct_1_0r2 describes itself as designed to work in Win 9X/ME.

I see compiling a wnaspi32.dll (e.g. via the wnaspi32.cpp for mingw gcc
attached) makes udfct_1_0r2 work in Win 2K/XP too.

I'm told Linux udfct_1_0r2 as yet works only on disk images ... such as
we create by default any time we substitute a loop device for a real
device.  But I see that wnaspi32.dll I compiled rests on top of gccscsi,
which by design talks thru Linux /dev/sg, not just Win/ Dos/ MacOF, so I
wonder if we are close to seeing udfct_1_0r2 run in linux.

I hear udfct_1_0r2 dumps only all the metadata of a disk, never the
metadata of a single file.

I of course welcome any correction/ elaboration you have to offer.

Pat LaVarre

P.S. Google at this moment no longer requires us to remember to say
Verifier rather than Test.  That is, to rediscover udfct_1_0r2 we now
can use the more concise link:

http://www.google.com/search?q=philips+udf+test



[-- Attachment #2: wnaspi32.cpp --]
[-- Type: text/x-c++, Size: 4590 bytes --]

/*
 * wnaspi32.cpp
 *
 * This source file for Win 2K/XP exists to make:
 *
 *	LoadLibrary("wnaspi32.dll");
 *
 * work enough like it did in Win 9X/ME to allow
 * trivial apps to function.
 *
 * Our "wnaspi32.h" appears unattributed, unlicensed,
 * and without copyright, precisely as it reached us
 * via Google.
 *
 * msdn.microsoft.com probably documents what our
 * history of pain has taught us of this interface.
 *
 * Bugs include:
 *
 *	Misbehaviour above 2 GiB per CDB.
 *
 *	Drive letter = ('C' + SRB_Target).
 *	No CloseHandle.
 *	CreateFile called til success, for each cdb sent to a drive letter.
 *
 *	Auto sense data echod out stderr, never passed back.
 *	No check for unused for the fields not copied, e.g. SRB_SenseLen.
 *
 *	x00 SC_HA_INQUIRY never becomes IOCTL_SCSI_GET_CAPABILITIES
 *	x02 SC_EXEC_SCSI_CMD never becomes IOCTL_SCSI_PASS_THROUGH_DIRECT
 *
 *	x80 SS_INVALID_CMD includes x08 SC_GETSET_TIMEOUTS
 *	x80 SS_INVALID_CMD includes x07 SC_RESCAN_SCSI_BUS
 *	x80 SS_INVALID_CMD includes x06 SC_GET_DISK_INFO
 *	x80 SS_INVALID_CMD includes x05 SC_SET_HA_PARMS
 *	x80 SS_INVALID_CMD includes x04 SC_RESET_DEV
 *	x80 SS_INVALID_CMD includes x03 SC_ABORT_SRB
 *	x80 SS_INVALID_CMD includes x01 SC_GET_DEV_TYPE
 */

#include "windows.h"
#include "ntddscsi.hpp"

#include <stddef.h>
#include <stdio.h>

#include "gccscsi.h"

#include "wnaspi32.hpp"

#define TARGET_COUNT (1 << 5) /* enough for 'C' .. 'Z' and one host */
int devs[TARGET_COUNT];

extern "C" {
extern DWORD GetASPI32SupportInfo(void);
extern DWORD SendASPI32Command(LPSRB v);
}

__declspec(dllexport)
extern DWORD GetASPI32SupportInfo(void)
{
	BYTE srbStatus = SS_NO_ADAPTERS; /* xE8 */
	BYTE haCount = ('Z' - 'A' + 1);
	srbStatus = SS_COMP; /* x01 */
	return ((srbStatus << 8) | haCount);
}

static BYTE ha_inquiry(SRB_HAInquiry * shai)
{
	WORD alignment_mask = (4 * Ki);
	BYTE may_count_data = 0x01;
	DWORD max_length = (64 * Ki);

	shai->HA_Count = 1;

	shai->HA_SCSI_ID = (TARGET_COUNT - 1);
	memset(&shai->HA_ManagerId[0], '\0', sizeof shai->HA_ManagerId);
	memset(&shai->HA_Identifier[0], '\0', sizeof shai->HA_Identifier);
	memset(&shai->HA_Unique[0], '\0', sizeof shai->HA_Unique);

	strncpy((char *) &shai->HA_ManagerId[0], "Wnaspi32.dll", sizeof shai->HA_ManagerId);
	strncpy((char *) &shai->HA_Identifier[0], "Slow SPT (not SPTD)", sizeof shai->HA_Identifier);

	* (WORD *) &shai->HA_Unique[0x00] = alignment_mask;
	* (BYTE *) &shai->HA_Unique[0x02] = may_count_data;
	* (BYTE *) &shai->HA_Unique[0x03] = TARGET_COUNT;
	* (DWORD *) &shai->HA_Unique[0x04] = max_length;

	return SS_COMP; /* x01 */
}

static BYTE exec_scsi_cmd(SRB_ExecSCSICmd * sesc)
{
	BYTE SRB_Target = sesc->SRB_Target;
	BYTE SRB_Lun = sesc->SRB_Lun;

	int letter = ('C' + SRB_Target);
	if (!(('A' <= letter) & (letter <= 'Z'))) return SS_NO_DEVICE; /* x82 */
	if (SRB_Lun != 0x00) return SS_NO_DEVICE; /* x82 */

	int dev = devs[SRB_Target];
	if (dev == 0) {
		char name[123] = "//./A:";
		sprintf(&name[0], "\\\\.\\%c:", letter);
		dev = sp_open(&name[0]);
	}
	if (dev == 0) return SS_NO_DEVICE; /* x82 */

	char * cdbChars = (char *) &sesc->CDBByte[0];
	int cdbLength = sesc->SRB_CDBLen;
	char * inChars = (char *) sesc->SRB_BufPointer;
	char * outChars = inChars;
	int maxLength = (int) sesc->SRB_BufLen;

	BYTE SRB_Flags = sesc->SRB_Flags;
	BYTE SRB_Flags_In_Out = (SRB_Flags & (SRB_DIR_OUT|SRB_DIR_IN)); /* x18 */
	if (maxLength == 0) {
		outChars = inChars = NULL;
	} else if (SRB_Flags_In_Out == SRB_DIR_IN) { /* x08 */
		outChars = NULL;
	} else if (SRB_Flags_In_Out == SRB_DIR_OUT) { /* x10 */
		inChars = NULL;
	} else {
		return SS_INVALID_SRB; /* xE0 */
	}

	int rc = sp_say(dev, cdbChars, cdbLength, outChars, inChars, maxLength);

	if (rc == 0) {
		sesc->SRB_HaStat = HASTAT_OK; /* x00 */
		sesc->SRB_TargStat = 0x00; /* good */
		return SS_COMP; /* x01 */
	}

	if (0 < rc) {
		sesc->SRB_HaStat = HASTAT_DO_DU; /* x12 */
		sesc->SRB_TargStat = 0x00; /* good */
		return SS_ERR; /* x04 */
	}

	sesc->SRB_HaStat = HASTAT_BUS_FREE; /* x13 */
	sesc->SRB_TargStat = 0x00; /* good */
	return SS_ERR; /* x04 */
}

__declspec(dllexport)
extern DWORD SendASPI32Command(LPSRB v)
{
	BYTE srbStatus = SS_INVALID_CMD; /* x80 */
	SRB_Abort * sa = (SRB_Abort *) v;
	if (sa->SRB_HaId != 0) {
		return SS_INVALID_HA;
	}
	switch (sa->SRB_Cmd) {
		case SC_HA_INQUIRY: /* x00 */
			srbStatus = ha_inquiry((SRB_HAInquiry *) v);
			break;
		case SC_EXEC_SCSI_CMD: /* x02 */
			srbStatus = exec_scsi_cmd((SRB_ExecSCSICmd *) v);
			break;
		default:
			break;
	}
	sa->SRB_Status = srbStatus;
	return srbStatus;
}

/* end of file */

Re: editable udf metadata

[Pat LaVarre]

[ BC linux_udf@hpesjro.fc.hp.com ]

> The web trail I know to the udfct_1_0r2 tool is:
> 
> http://www.google.com/search?q=philips+udf+verifier
> http://www.google.com/search?q=philips+udf+verifier&btnI=1
> http://www.extra.research.philips.com/udf/
> http://www.extra.research.philips.com/udf/download.html
> 
> I see udfct_1_0r2 describes itself as designed to work in Win 9X/ME.
> 
> I see compiling a wnaspi32.dll (e.g. via the wnaspi32.cpp for mingw gcc
> attached) makes udfct_1_0r2 work in Win 2K/XP too.
> 
> I'm told Linux udfct_1_0r2 as yet works only on disk images ...

I saw this claim kindly challenged here, I'm now beginning to believe
the challenge.  I have now found:

http://www.extra.research.philips.com/udf/download/udfct_1_0r2.tar.gz
FAQ.TXT
Binaries with SCSI/Atapi support are only present for Windows and Linux

Possibly I was just remembering udfct as yet another example of a lab
tool that chokes because Mac OS X has no root-privileged SCSI pass thru.

Pat LaVarre

Re: editable udf metadata

[Pat LaVarre]

Correction.

We mean to say udfct in Linux needs no SCSI pass thru, right?

We mean to say udfct only needs a raw block device?

Mac OS X has those too, so Mac OS X could work if we volunteer a patch?

Pat LaVarre

$ cd udfct_1.0r2
$ less FAQ.txt
$ cd ../udfct_1.0r2/src/udf_tester
$ make
...
$
$ ./udf_test /dev/sg0
UDF Conformance Testing Application
(c) Koninklijke Philips Electronics N.V. 1999-2003
Application version: 1.0r2
UCT Core version   : 1.0r2
...
No image configuration file /dev/sg0.cfg 
 
  Error: Image file chunk byte size: 0,
-        expected: not zero, at most (1048575 * 2048), and
-        an integral multiple of the block size (2048).
-   for: /dev/sg0
 
Cannot create image device
...
fatal error: Exit code 2, quit.
...
$
$ ./udf_test -show-scsi
...
Unknown, misplaced or unused command line arguments:
        -show-scsi
fatal error: Exit code 2, quit.
...
$
$ sudo dd of=/dev/scd0 bs=2K count=1 if=/dev/zero
1+0 records in
1+0 records out
$
$ mkudffs /dev/scd0
trying to change type of multiple extents
$
$ sudo mkudffs /dev/scd0
start=0, blocks=16, type=RESERVED
...
start=524287, blocks=1, type=ANCHOR
$
$ sudo udf_test /dev/scd0 >1
$ sudo udf_test /dev/scd0 >2
$ diff 1 2
50c50
<       Start time   : 2003-10-21 17:14:31 -06:00 (west of UTC)
---
>       Start time   : 2003-10-21 17:14:33 -06:00 (west of UTC)
$
$ grep -i 'note:' 1 | wc -l
      0
$ grep -i 'warning:' 1 | wc -l
      0
$ grep -i 'error:' 1 | wc -l
      1
$
$ grep -i -A 6 'error:' 1
        EFE  28  icbtag error: parentICBLocation: (34,0),
-                       recommended: (0,0) for strategy type 4.
-                Icbtag Parent ICB Location shall not point to itself
-                or to a higher parent in the directory hierarchy.
-                ECMA 3rd edition 4/8.10.1, 4/14.6.7, UDF 2.3.5.3.
-                icbtag: ECMA 4/14.6.*, 4/A.5, UDF 2.3.5.*, 6.6
-                name: "lost+found"
$

Re: editable udf metadata

[Pat LaVarre]

> We mean to say udfct in Linux needs no SCSI pass thru, right?
> We mean to say udfct only needs a raw block device?
> Mac OS X has those too, so Mac OS X could work if we volunteer a patch?

No explicit answers yet.

But here now in summary we do have four UDF FAQ's answered, per request
kindly received offline:

/// Contents

1: Where on the web does the fsck for linux udf.ko appear?
2: Why does an erased disk fail udf fsck?
3: Where on the web does an already `make`able udf fsck appear?
4: Why doesn't udfct_1_0r2/ understand `./configure ; make`?

/// 1: Where on the web does the fsck for linux udf.ko appear?

As the udffsck/ of:

$ export CVSROOT=:pserver:anonymous@cvs.sourceforge.net:/cvsroot/linux-udf
$ cvs login
(Logging in to anonymous@cvs.sourceforge.net)
CVS password:
$ cvs co udftools
...

/// 2: Why does an erased disk fail udf fsck?

It doesn't, if you erased your disk correctly.

The one "error:|warning:|note:" you can expect is "EFE  28  icbtag
error: parentICBLocation: (10,0)," if you erased your disk via the
mkudffs/mkudffs of 2002 Feb 9 1.0.0b2 udftools freely offered as the
"latest file release" of:
http://sourceforge.net/projects/linux-udf/

If you want no "error:|warning:|note:", then instead you can try the
mkudffs/mkudffs you get via CVS.

/// 3: Where on the web does an already `make`able udf fsck appear?

We haven't yet published how to make the cvs udf fsck.

At the end of the web trail:

http://www.google.com/search?q=philips+udf+test
http://www.google.com/search?q=philips+udf+verifier&btnI=1
http://www.extra.research.philips.com/udf/
http://www.extra.research.philips.com/udf/download.html
http://www.extra.research.philips.com/udf/download/udfct_1_0r2.tar.gz

The src/udf_tester/udf_test you make is the "Philips UDF Verifier" of
Gerrit Scholls et al.

/// 4: Why doesn't udfct_1_0r2/ understand `./configure ; make`?

I have no idea.  But a tty log of what worked for me follows:

$ cd udfct_1.0r2
$ less FAQ.txt
$ cd src/udf_tester
$ make
...
$
$ ./udf_test /dev/sg0
UDF Conformance Testing Application
(c) Koninklijke Philips Electronics N.V. 1999-2003
Application version: 1.0r2
UCT Core version   : 1.0r2
...
No image configuration file /dev/sg0.cfg 
 
  Error: Image file chunk byte size: 0,
-        expected: not zero, at most (1048575 * 2048), and
-        an integral multiple of the block size (2048).
-   for: /dev/sg0
 
Cannot create image device
...
fatal error: Exit code 2, quit.
...
$
$ ./udf_test -show-scsi
...
Unknown, misplaced or unused command line arguments:
        -show-scsi
fatal error: Exit code 2, quit.
...
$
$ sudo dd of=/dev/scd0 bs=2K count=1 if=/dev/zero
1+0 records in
1+0 records out
$
$ mkudffs /dev/scd0
trying to change type of multiple extents
$
$ sudo mkudffs /dev/scd0
start=0, blocks=16, type=RESERVED
...
start=524287, blocks=1, type=ANCHOR
$
$ sudo udf_test /dev/scd0 >1
$ sudo udf_test /dev/scd0 >2
$ diff 1 2
50c50
<       Start time   : 2003-10-21 17:14:31 -06:00 (west of UTC)
---
>       Start time   : 2003-10-21 17:14:33 -06:00 (west of UTC)
$
$ grep -i 'note:' 1 | wc -l
      0
$ grep -i 'warning:' 1 | wc -l
      0
$ grep -i 'error:' 1 | wc -l
      1
$
$ grep -i -A 6 'error:' 1
        EFE  28  icbtag error: parentICBLocation: (34,0),
-                       recommended: (0,0) for strategy type 4.
-                Icbtag Parent ICB Location shall not point to itself
-                or to a higher parent in the directory hierarchy.
-                ECMA 3rd edition 4/8.10.1, 4/14.6.7, UDF 2.3.5.3.
-                icbtag: ECMA 4/14.6.*, 4/A.5, UDF 2.3.5.*, 6.6
-                name: "lost+found"
$

Re: editable udf metadata

[Pat LaVarre]

> 3: Where on the web does an already `make`able udf fsck appear?
> 
> We haven't yet published how to make the cvs udf fsck.
> ...

Correction.  By now we have explained how to make.  Now the answer is
merely:

./bootstrap
./configure
make

However, as yet, not counting comments, the udffsck/udffsck tool we're
making is merely:

int main(int argc, char *argv[])
{
	return 0;
}

So as yet the FAQ directing you to udfct udf_test (aka the Philips UDF
Verifier) still applies.

Pat LaVarre

editable udf metadata

[Pat LaVarre]

Kindly offline I was asked to summarise & elaborate:

1)

The udfct udf_test (aka the Philips UDF Verifier) encourages you to
fetch udftools via cvs rather than via http of sourceforge, per such
procedures as:

http://marc.theaimsgroup.com/?l=linux-fsdevel&m=106692617907993

List:     linux-fsdevel
Subject:  Re: editable udf metadata
Date:     2003-10-23 16:06:26

2)

The cvs udftools you get that way didn't change between 2003-10-24 and
2003-11-26.

3)

A snapshot of the cvs udftools, by design likely to fall behind just as
http sourceforge does, appears at:

ftp://members.aol.com/plscsi/linux/

4)

When fetching via plscsi/ you can say http rather than ftp if you like:

http://members.aol.com/plscsi/linux/

Pat LaVarre