Compatible fstat()

Compatible fstat()

[linux-os (Dick Johnson)]

Hello,

The Linux fstat() doesn't return any information number of blocks,
or the byte-length of a physical hard disk.

       Device name : /dev/hda
     Device st_dev : 00000301
    Device st_rdev : 00000300
    Device st_size : 0
  Device st_blocks : 0
Device st_blksize : 4096
       Device name : /dev/sda
     Device st_dev : 00000301
    Device st_rdev : 00000800
    Device st_size : 0
  Device st_blocks : 0
Device st_blksize : 4096
       Device name : /dev/sdb
     Device st_dev : 00000301
    Device st_rdev : 00000810
    Device st_size : 0
  Device st_blocks : 0
Device st_blksize : 4096
       Device name : /dev/sdc
     Device st_dev : 00000301
    Device st_rdev : 00000820
    Device st_size : 0
  Device st_blocks : 0
Device st_blksize : 4096

So, in a PORTABLE way, how do I find the block device with a
certain size, perhaps even the largest or smallest? I need to
have a customer plug in an external drive (USB) and be able
to determine if it's big enough for a raw-data write (no
file-system).

Cheers,
Dick Johnson
Penguin : Linux version 2.6.13.4 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.
.

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Compatible fstat()

[Parag Warudkar]

On Nov 8, 2005, at 10:48 AM, linux-os ((Dick Johnson)) wrote:

> The Linux fstat() doesn't return any information number of blocks,
> or the byte-length of a physical hard disk.

I don't think (f)stat returns size and blocks information about a  
block device on any UNIX platform.

But I don't know for sure how to get it - perhaps ioctl on the  
device? BLKGETSIZE?

Parag

Re: Compatible fstat()

[Al Viro]

On Tue, Nov 08, 2005 at 12:10:25PM -0500, Parag Warudkar wrote:
> 
> On Nov 8, 2005, at 10:48 AM, linux-os ((Dick Johnson)) wrote:
> 
> >The Linux fstat() doesn't return any information number of blocks,
> >or the byte-length of a physical hard disk.
> 
> I don't think (f)stat returns size and blocks information about a  
> block device on any UNIX platform.
> 
> But I don't know for sure how to get it - perhaps ioctl on the  
> device? BLKGETSIZE?

	fd = open(bdev, O_RDONLY);
	lseek(fd, SEEK_END, 0);
	size = lseek(fd, SEEK_SET, 0);
	close(fd);

i.e. same as for regular files.  Won't be portable, though...

Re: Compatible fstat()

[linux-os (Dick Johnson)]

On Tue, 8 Nov 2005, Parag Warudkar wrote:

>
> On Nov 8, 2005, at 10:48 AM, linux-os ((Dick Johnson)) wrote:
>
>> The Linux fstat() doesn't return any information number of blocks,
>> or the byte-length of a physical hard disk.
>
> I don't think (f)stat returns size and blocks information about a
> block device on any UNIX platform.
>
> But I don't know for sure how to get it - perhaps ioctl on the
> device? BLKGETSIZE?
>
> Parag
>

I'll see if it works on my Sun. Thanks.


Cheers,
Dick Johnson
Penguin : Linux version 2.6.13.4 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.
.

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Compatible fstat()

[linux-os (Dick Johnson)]

On Tue, 8 Nov 2005, Al Viro wrote:

> On Tue, Nov 08, 2005 at 12:10:25PM -0500, Parag Warudkar wrote:
>>
>> On Nov 8, 2005, at 10:48 AM, linux-os ((Dick Johnson)) wrote:
>>
>>> The Linux fstat() doesn't return any information number of blocks,
>>> or the byte-length of a physical hard disk.
>>
>> I don't think (f)stat returns size and blocks information about a
>> block device on any UNIX platform.
>>
>> But I don't know for sure how to get it - perhaps ioctl on the
>> device? BLKGETSIZE?
>
> 	fd = open(bdev, O_RDONLY);
> 	lseek(fd, SEEK_END, 0);
> 	size = lseek(fd, SEEK_SET, 0);
> 	close(fd);
>
> i.e. same as for regular files.  Won't be portable, though...
>
I think lseek has more of a chance than ioctl(), though.
Thanks.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.13.4 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.
.

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Compatible fstat()

[Parag Warudkar]

On Nov 8, 2005, at 12:22 PM, Al Viro wrote:
>
> 	fd = open(bdev, O_RDONLY);
> 	lseek(fd, SEEK_END, 0);
> 	size = lseek(fd, SEEK_SET, 0);
> 	close(fd);
>
> i.e. same as for regular files.  Won't be portable, though...

For some reason this didn't work for bdev == "/dev/hda" - Size is  
returned as 0..

Parag

Re: Compatible fstat()

[Parag Warudkar]

On Nov 8, 2005, at 12:53 PM, linux-os ((Dick Johnson)) wrote:
>>
>
> I'll see if it works on my Sun. Thanks.
>

Sun? ioctls being what they are it most likely won't work. Since  
there doesn't seem to be a portable way of doing this my response was  
linux-centric. (I know at least fstat doesn't give what you are  
looking for on Linux and *BSD)

Parag

Re: Compatible fstat()

[linux-os (Dick Johnson)]

On Tue, 8 Nov 2005, Parag Warudkar wrote:

>
> On Nov 8, 2005, at 12:22 PM, Al Viro wrote:
>>
>> 	fd = open(bdev, O_RDONLY);
>> 	lseek(fd, SEEK_END, 0);
>> 	size = lseek(fd, SEEK_SET, 0);
>> 	close(fd);
>>
>> i.e. same as for regular files.  Won't be portable, though...
>
> For some reason this didn't work for bdev == "/dev/hda" - Size is
> returned as 0..
>

I think it's just a syntax error...

>> 	size = lseek(fd, 0, SEEK_SET);
                             |___________  Whence at the end

... will do fine on three different systems so far.


Cheers,
Dick Johnson
Penguin : Linux version 2.6.13.4 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.
.

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Compatible fstat()

[Parag Warudkar]

>
> I think it's just a syntax error...
>
>>> 	size = lseek(fd, 0, SEEK_SET);
>                              |___________  Whence at the end
>
> ... will do fine on three different systems so far.
>

Yeah I corrected that before trying but still didn't work on Debian  
(2.6.8 kernel)...
Running root, open successful but size is always zero - Strange..

Parag

Re: Compatible fstat()

[linux-os (Dick Johnson)]

On Tue, 8 Nov 2005, Parag Warudkar wrote:

>
>>
>> I think it's just a syntax error...
>>
>>>> 	size = lseek(fd, 0, SEEK_SET);
>>                              |___________  Whence at the end
>>
>> ... will do fine on three different systems so far.
>>
>
> Yeah I corrected that before trying but still didn't work on Debian
> (2.6.8 kernel)...
> Running root, open successful but size is always zero - Strange..
>
> Parag

Also found that the returned value was -1 and errno was EOVERFLOW.
So, that doesn't work either!

Cheers,
Dick Johnson
Penguin : Linux version 2.6.13.4 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.
.

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Compatible fstat()

[Theodore Ts'o]

On Tue, Nov 08, 2005 at 05:22:44PM +0000, Al Viro wrote:
> On Tue, Nov 08, 2005 at 12:10:25PM -0500, Parag Warudkar wrote:
> > 
> > On Nov 8, 2005, at 10:48 AM, linux-os ((Dick Johnson)) wrote:
> > 
> > >The Linux fstat() doesn't return any information number of blocks,
> > >or the byte-length of a physical hard disk.
> > 
> > I don't think (f)stat returns size and blocks information about a  
> > block device on any UNIX platform.
> > 
> > But I don't know for sure how to get it - perhaps ioctl on the  
> > device? BLKGETSIZE?
> 
> 	fd = open(bdev, O_RDONLY);
> 	lseek(fd, SEEK_END, 0);
> 	size = lseek(fd, SEEK_SET, 0);
> 	close(fd);
> 
> i.e. same as for regular files.  Won't be portable, though...

As I recall, this didn't always work; e2fsprogs falls back to using a
binary search using SEEK_SET to find the device size.  Folks who need
to do this should look at lib/ext2fs/getsize.c in the e2fsprogs
sources; it has ioctl's and other procedures for determining block
device size for Linux, *BSD, MacOSX, as well a generalized binary
search algorithm.

						- Ted

Re: Compatible fstat()

[Parag Warudkar]

On Nov 8, 2005, at 1:49 PM, Theodore Ts'o wrote:

> e2fsprogs falls back to using a
> binary search using SEEK_SET to find the device size.

Binary search of what? I tried to read the relevant code in getsize.c  
but apart from suspecting that the binary search thing might be  
specific to ext2fs I didn't quite understand what's going on in the  
code.  (Will it work irrespective of the file system presence on the  
device?)

Thanks
Parag

Re: Compatible fstat()

[Bob Copeland]

> > Yeah I corrected that before trying but still didn't work on Debian
> > (2.6.8 kernel)...
> > Running root, open successful but size is always zero - Strange..
> >
> > Parag
>
> Also found that the returned value was -1 and errno was EOVERFLOW.
> So, that doesn't work either!

Isn't this just because the device size is > 2**32?  What if you use fseeko(3)
and #define _FILE_OFFSET_BITS 64?

Okay, still not portable and there is probably a better way that doesn't rely
on such nonsense.  For example, since you have a minimum size in mind,
just seek that much and see if it works - you don't really need to know the
whole disk size for that.  Or figure out the best way on all of your platforms
and abstract it.

-Bob

Re: Compatible fstat()

[linux-os (Dick Johnson)]

On Tue, 8 Nov 2005, Bob Copeland wrote:

>>> Yeah I corrected that before trying but still didn't work on Debian
>>> (2.6.8 kernel)...
>>> Running root, open successful but size is always zero - Strange..
>>>
>>> Parag
>>
>> Also found that the returned value was -1 and errno was EOVERFLOW.
>> So, that doesn't work either!
>
> Isn't this just because the device size is > 2**32?  What if you use fseeko(3)
> and #define _FILE_OFFSET_BITS 64?
>
> Okay, still not portable and there is probably a better way that doesn't rely
> on such nonsense.  For example, since you have a minimum size in mind,
> just seek that much and see if it works - you don't really need to know the
> whole disk size for that.  Or figure out the best way on all of your platforms
> and abstract it.
>
> -Bob

Well if I could __count__ on EOVERFLOW meaning there was plenty of
room, it would be okay. Also, if I could count on walking-up-the
ladder of some SEEK_SET code that would be good also.

Anyway. I'm still checking. It sure would have been helpful
if fstat returned the number of blocks and the block-size for
the device (as it implied by the existance of those members).
Unfortunately the only POSIX requirement is for the st_*time,
st_mode, st_ini, st_dev, and st_*id fields to contain meaningful
values, damn.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.13.4 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.
.

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Compatible fstat()

[Parag Warudkar]

On Nov 8, 2005, at 2:39 PM, Bob Copeland wrote:

> Isn't this just because the device size is > 2**32?  What if you  
> use fseeko(3)
> and #define _FILE_OFFSET_BITS 64?
>

Yep. I got it to return the correct hard disk size (17Gb) using  
lseek64 and
#define _LARGEFILE64_SOURCE
#define _FILE_OFFSET_BITS 64

Here is what I did
-------------------------------------------------
#define _LARGEFILE64_SOURCE
#define _FILE_OFFSET_BITS 64
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>

int main()
{
         int f;
         off64_t off=0;
         f = open("/dev/hda", O_RDONLY );
         if(f <= 0){
                 perror("open");
                 exit(0);
         }
         off = lseek64(f, 0, SEEK_SET);
         off = lseek64(f, 0, SEEK_END);
         perror("llseek");
         printf ("Size %lld\n", off);
         close(f);
         return 0;
}



> Okay, still not portable and there is probably a better way that  
> doesn't rely
> on such nonsense.  For example, since you have a minimum size in mind,
> just seek that much and see if it works - you don't really need to  
> know the
> whole disk size for that.  Or figure out the best way on all of  
> your platforms
> and abstract it.

Now this gives me a clue what Ted was referring to as binary search  
in e2fsprogs.
If nothing works this should work portably I guess.

Thanks
Parag 

Re: Compatible fstat()

[Bill Davidsen]

Parag Warudkar wrote:
> 
> On Nov 8, 2005, at 2:39 PM, Bob Copeland wrote:
> 
>> Isn't this just because the device size is > 2**32?  What if you  use 
>> fseeko(3)
>> and #define _FILE_OFFSET_BITS 64?
>>
> 
> Yep. I got it to return the correct hard disk size (17Gb) using  lseek64 
> and
> #define _LARGEFILE64_SOURCE
> #define _FILE_OFFSET_BITS 64
> 
> Here is what I did
> -------------------------------------------------
> #define _LARGEFILE64_SOURCE
> #define _FILE_OFFSET_BITS 64
> #include <stdio.h>
> #include <unistd.h>
> #include <fcntl.h>
> 
> int main()
> {
>         int f;
>         off64_t off=0;
Why is this initialized?
>         f = open("/dev/hda", O_RDONLY );
>         if(f <= 0){
>                 perror("open");
>                 exit(0);
>         }
>         off = lseek64(f, 0, SEEK_SET);
Why do this? it always returns zero.
>         off = lseek64(f, 0, SEEK_END);
>         perror("llseek");
>         printf ("Size %lld\n", off);
>         close(f);
>         return 0;
> }
> 
RETURN VALUE
   Upon successful completion, lseek returns the resulting offset
   location as measured in bytes from the beginning of the  file.
   Otherwise,  a  value  of  (off_t)-1 is returned and errno is
   set to indicate the error.


-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

Re: Compatible fstat()

[Parag Warudkar]

On Nov 8, 2005, at 5:57 PM, Bill Davidsen wrote:

> Parag Warudkar wrote:
>> On Nov 8, 2005, at 2:39 PM, Bob Copeland wrote:
>>> Isn't this just because the device size is > 2**32?  What if you   
>>> use fseeko(3)
>>> and #define _FILE_OFFSET_BITS 64?
>>>
>> Yep. I got it to return the correct hard disk size (17Gb) using   
>> lseek64 and
>> #define _LARGEFILE64_SOURCE
>> #define _FILE_OFFSET_BITS 64
>> Here is what I did
>> -------------------------------------------------
>> #define _LARGEFILE64_SOURCE
>> #define _FILE_OFFSET_BITS 64
>> #include <stdio.h>
>> #include <unistd.h>
>> #include <fcntl.h>
>> int main()
>> {
>>         int f;
>>         off64_t off=0;
> Why is this initialized?
>>         f = open("/dev/hda", O_RDONLY );
>>         if(f <= 0){
>>                 perror("open");
>>                 exit(0);
>>         }
>>         off = lseek64(f, 0, SEEK_SET);
> Why do this? it always returns zero.
>>         off = lseek64(f, 0, SEEK_END);
>>         perror("llseek");
>>         printf ("Size %lld\n", off);
>>         close(f);
>>         return 0;
>> }
> RETURN VALUE
>   Upon successful completion, lseek returns the resulting offset
>   location as measured in bytes from the beginning of the  file.
>   Otherwise,  a  value  of  (off_t)-1 is returned and errno is
>   set to indicate the error.
>

You took it a little too seriously! It was from the scribble-and- 
shuffle-till-it-works department ;) Sole purpose was to figure out a  
way to print the disk size from the device - some how that is!

But thanks for pointing out anyway - it doesn't hurt to be correct no  
matter what the purpose is.

Parag

Re: Compatible fstat()

[Theodore Ts'o]

On Tue, Nov 08, 2005 at 02:12:24PM -0500, Parag Warudkar wrote:
> 
> On Nov 8, 2005, at 1:49 PM, Theodore Ts'o wrote:
> 
> >e2fsprogs falls back to using a
> >binary search using SEEK_SET to find the device size.
> 
> Binary search of what? 

Of the device size; it doubles the guessed size of the disk until
lseek+read returns an error, and then uses binary search to figure out
the size of the disk.  I did this because it works on pretty much any
OS.

>  I tried to read the relevant code in getsize.c  
> but apart from suspecting that the binary search thing might be  
> specific to ext2fs I didn't quite understand what's going on in the  
> code.  (Will it work irrespective of the file system presence on the  
> device?)

Yes, it works irrespective of what's on the disk.  In fact, if the
Linux-specific ioctl's are not available, it's what will be used by
mke2fs to figure out the size of the device.

					- Ted

Re: Compatible fstat()

[Bill Davidsen]

Parag Warudkar wrote:

>
> On Nov 8, 2005, at 5:57 PM, Bill Davidsen wrote:
>
>> Parag Warudkar wrote:
>>
>>> On Nov 8, 2005, at 2:39 PM, Bob Copeland wrote:
>>>
>>>> Isn't this just because the device size is > 2**32?  What if you   
>>>> use fseeko(3)
>>>> and #define _FILE_OFFSET_BITS 64?
>>>>
>>> Yep. I got it to return the correct hard disk size (17Gb) using   
>>> lseek64 and
>>> #define _LARGEFILE64_SOURCE
>>> #define _FILE_OFFSET_BITS 64
>>> Here is what I did
>>> -------------------------------------------------
>>> #define _LARGEFILE64_SOURCE
>>> #define _FILE_OFFSET_BITS 64
>>> #include <stdio.h>
>>> #include <unistd.h>
>>> #include <fcntl.h>
>>> int main()
>>> {
>>>         int f;
>>>         off64_t off=0;
>>
>> Why is this initialized?
>>
>>>         f = open("/dev/hda", O_RDONLY );
>>>         if(f <= 0){
>>>                 perror("open");
>>>                 exit(0);
>>>         }
>>>         off = lseek64(f, 0, SEEK_SET);
>>
>> Why do this? it always returns zero.
>>
>>>         off = lseek64(f, 0, SEEK_END);
>>>         perror("llseek");
>>>         printf ("Size %lld\n", off);
>>>         close(f);
>>>         return 0;
>>> }
>>
>> RETURN VALUE
>>   Upon successful completion, lseek returns the resulting offset
>>   location as measured in bytes from the beginning of the  file.
>>   Otherwise,  a  value  of  (off_t)-1 is returned and errno is
>>   set to indicate the error.
>>
>
> You took it a little too seriously! It was from the scribble-and- 
> shuffle-till-it-works department ;) Sole purpose was to figure out a  
> way to print the disk size from the device - some how that is!
>
> But thanks for pointing out anyway - it doesn't hurt to be correct no  
> matter what the purpose is. 

You could look at how blockdev does it, the --getss (sector size) and 
--getsize (size in sectors) seem to work, based on trying it on a total 
of two machines ;-) Even gives correct size for software RAID arrays!

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979