Few questions

Few questions

[Michael Makuch]

I realize this is the developers list and though I am a developer I'm 
not a developer
of linux raid, but I can find no other source of answers to these questions:

I've been using linux software raid (5) for a couple of years, having 
recently uped
to the 2.6.23 kernel (FC7, was previously on FC5). I just noticed that
my /proc/mdstat shows

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 etherd/e0.0[0] etherd/e0.2[9](S) etherd/e0.9[8] 
etherd/e0.8[7] etherd/e0.7[6] etherd/e0.6[5] etherd/e0.5[4] 
etherd/e0.4[3] etherd/e0.3[2] etherd/e0.1[1]
      3907091968 blocks level 5, 64k chunk, algorithm 2 [9/9] [UUUUUUUUU]
      [============>........]  resync = 64.5% (315458352/488386496) 
finish=2228.0min speed=1292K/sec
unused devices: <none>

and I have no idea where the raid6 came from. The only thing I've found 
on raid6
is a wikipedia.org page, nothing on 
http://tldp.org/HOWTO/Software-RAID-HOWTO.html

So my questions are:

- Is raid6 documented anywhere? If so, where? I'd like to take advantage 
of it if
it's really there.
- Why does my array (which I configured as raid5) have personalities of
raid6 (I can understand why raid4 would be there)?
- Is this a.o.k for a raid5 array?

Thanks

RE: Few questions

[Guy Watkins]

man md
man mdadm

I use RAID6.  Happy with it so far, but haven't had a disk failure yet.
RAID5 sucks because if you have 1 failed disk and 1 bad block on any other
disk, you are hosed.

Hope that helps.

} -----Original Message-----
} From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
} owner@vger.kernel.org] On Behalf Of Michael Makuch
} Sent: Friday, December 07, 2007 7:12 PM
} To: linux-raid@vger.kernel.org
} Subject: Few questions
} 
} I realize this is the developers list and though I am a developer I'm
} not a developer
} of linux raid, but I can find no other source of answers to these
} questions:
} 
} I've been using linux software raid (5) for a couple of years, having
} recently uped
} to the 2.6.23 kernel (FC7, was previously on FC5). I just noticed that
} my /proc/mdstat shows
} 
} $ cat /proc/mdstat
} Personalities : [raid6] [raid5] [raid4]
} md0 : active raid5 etherd/e0.0[0] etherd/e0.2[9](S) etherd/e0.9[8]
} etherd/e0.8[7] etherd/e0.7[6] etherd/e0.6[5] etherd/e0.5[4]
} etherd/e0.4[3] etherd/e0.3[2] etherd/e0.1[1]
}       3907091968 blocks level 5, 64k chunk, algorithm 2 [9/9] [UUUUUUUUU]
}       [============>........]  resync = 64.5% (315458352/488386496)
} finish=2228.0min speed=1292K/sec
} unused devices: <none>
} 
} and I have no idea where the raid6 came from. The only thing I've found
} on raid6
} is a wikipedia.org page, nothing on
} http://tldp.org/HOWTO/Software-RAID-HOWTO.html
} 
} So my questions are:
} 
} - Is raid6 documented anywhere? If so, where? I'd like to take advantage
} of it if
} it's really there.
} - Why does my array (which I configured as raid5) have personalities of
} raid6 (I can understand why raid4 would be there)?
} - Is this a.o.k for a raid5 array?
} 
} Thanks
} -
} To unsubscribe from this list: send the line "unsubscribe linux-raid" in
} the body of a message to majordomo@vger.kernel.org
} More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Few questions

[David Greaves]

Guy Watkins wrote:
> man md
> man mdadm
and
http://linux-raid.osdl.org/index.php/Main_Page

:)

Re: Few questions

[Michael Makuch]

Guy Watkins wrote:
> man md
> man mdadm
>
> I use RAID6.  Happy with it so far, but haven't had a disk failure yet.
> RAID5 sucks because if you have 1 failed disk and 1 bad block on any other
> disk, you are hosed.
>
> Hope that helps.
I can't believe I've been using a raid array for 2 years and didn't know 
'man md' was there. I've
lived and breathed off of Software-RAID-HOWTO.html but it never mentions it.

Just a suggestion to whom it may concern: it might be nice to mention 
'man md' in the Howto as well
as the wiki pages.

Thanks

Mike

Re: Few questions

[Corey Hickey]

Michael Makuch wrote:
> I realize this is the developers list and though I am a developer I'm 
> not a developer
> of linux raid, but I can find no other source of answers to these questions:

Don't worry; it's a user list too.

> $ cat /proc/mdstat
> Personalities : [raid6] [raid5] [raid4]
> md0 : active raid5 etherd/e0.0[0] etherd/e0.2[9](S) etherd/e0.9[8] 
> etherd/e0.8[7] etherd/e0.7[6] etherd/e0.6[5] etherd/e0.5[4] 
> etherd/e0.4[3] etherd/e0.3[2] etherd/e0.1[1]
>       3907091968 blocks level 5, 64k chunk, algorithm 2 [9/9] [UUUUUUUUU]
>       [============>........]  resync = 64.5% (315458352/488386496) 
> finish=2228.0min speed=1292K/sec
> unused devices: <none>
> 
> and I have no idea where the raid6 came from.

As far as I understand, the "Personalities" line just shows what RAID
capabilities are compiled into the kernel (and loaded, if modules). For
example, even though I'm only using raid0, I have:

-----------------------------------------------------------------
$ cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md0 : active raid0 sdc[0] sdb[1]
      976772992 blocks 64k chunks

unused devices: <none>
-----------------------------------------------------------------

-Corey

/proc/mdstat docs (was Re: Few questions)

[David Greaves]

Michael Makuch wrote:
> So my questions are:
...
> - Is this a.o.k for a raid5 array?

So I realised that /proc/mdstat isn't documented too well anywhere...

http://linux-raid.osdl.org/index.php/Mdstat

Comments welcome...

David

Re: /proc/mdstat docs (was Re: Few questions)

[Raz]

many thanks . david.
It is very useful.

On 12/8/07, David Greaves <david@dgreaves.com> wrote:
> Michael Makuch wrote:
> > So my questions are:
> ...
> > - Is this a.o.k for a raid5 array?
>
> So I realised that /proc/mdstat isn't documented too well anywhere...
>
> http://linux-raid.osdl.org/index.php/Mdstat
>
> Comments welcome...
>
> David
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


-- 
Raz

Re: /proc/mdstat docs (was Re: Few questions)

[Michael Makuch]

David Greaves wrote:
> Michael Makuch wrote:
>   
>> So my questions are:
>>     
> ...
>   
>> - Is this a.o.k for a raid5 array?
>>     
>
> So I realised that /proc/mdstat isn't documented too well anywhere...
>
> http://linux-raid.osdl.org/index.php/Mdstat
>
> Comments welcome...
>
> David
>   

One thing, in the section "md device line" you describe how to identify 
spare devices,
but you didn't mention the "(S)" which appears after a spare device, at 
least it does on mine:

# uname -a
Linux pecan.makuch.org 2.6.23.1-21.fc7 #1 SMP Thu Nov 1 21:09:24 EDT 
2007 i686 i686 i386 GNU/Linux

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 etherd/e0.0[0] etherd/e0.2[9](S) etherd/e0.9[8] 
etherd/e0.8[7] etherd/e0.7[6] etherd/e0.6[5] etherd/e0.5[4] 
etherd/e0.4[3] etherd/e0.3[2] etherd/e0.1[1]
      3907091968 blocks level 5, 64k chunk, algorithm 2 [9/9] [UUUUUUUUU]
      [==================>..]  resync = 91.0% (444527040/488386496) 
finish=560.9min speed=1301K/sec
unused devices: <none>


Unless that means something else??? But e0.2 is my spare so I'm just 
assuming "(S)" means spare!

Thanks
Mike

Few Questions

[Arya, Manish Kumar]

Hi,

   I am new to Git. Earlier I have configured svn with LDAP auth and svnwebclient.

I want to have following with Git

- LDAP and ssh authentication.

- checkin and checkout using web interface and ssh

- when ever someone checkin something then a email should be send to a email address (a mailing list)

please let me know how to do this with Git

-Manish 


      

Re: Few Questions

[Matthieu Moy]

"Arya, Manish Kumar" <m.arya@yahoo.com> writes:

> Hi,
>
>    I am new to Git. Earlier I have configured svn with LDAP auth and svnwebclient.
>
> I want to have following with Git
>
> - LDAP and ssh authentication.

AFAIK, there isn't any authentication mechanism built into Git.
Instead, Git relies on existing (proven, reliable, ...) mechanisms.
SSH authentication is what you get when accessing a repository with
e.g. git clone ssh://host.com/path/to/repo (either you have a full
ssh shell access on the server, or you can restrict the access with
git-shell to allow only basic git operations on the server).

There's probably a way to let your server use LDAP for authentication
when using SSH, but that's independant from Git (and I'm helpless
here).

> - checkin and checkout using web interface and ssh

Gitweb for the web interface. "checkin" and "checkout" have different
meanings depending on the tool, so I'm not sure I understand the
question correctly.

In Git, the equivalent of "checkout" for centralized VCS would be
"clone" (i.e. get a local working tree for a remote repository, but
Git also duplicates the history), see above, it works straigtforwardly
through SSH. I don't think you can do it from a web interface, but I
don't understand what would be the point in doing it.

> - when ever someone checkin something then a email should be send to
> a email address (a mailing list)

With a hook. You probably don't want to have this as the commit hook,
since the advantage of Git is to make "commit" a local, somehow
private operation, and to distinguish it from "push" (which somehow
means "publish", "show to the rest of the world"). So sending email
when some server receives the new revisions is sensible, this is the
post-receive hook.

These can help:

http://git.kernel.org/?p=git/git.git;a=blob;f=contrib/hooks/post-receive-email;h=28a3c0e46ecf9951f3f42a025a288a65c70e0424;hb=HEAD
http://source.winehq.org/git/tools.git/?a=blob;f=git-notify;hb=HEAD

-- 
Matthieu

Re: Few Questions

[Jakub Narebski]

"Arya, Manish Kumar" <m.arya@yahoo.com> writes:

> Hi,
> 
> I am new to Git. Earlier I have configured svn with LDAP auth and
> svnwebclient.
> 
> I want to have following with Git
> 
> - LDAP and ssh authentication.
> 

Instead of inventing (and failing) its own protocol and its own
authentication git uses established solutions for authentication: SSH
for "smart" server, and WebDAV for push via (currently only "dumb")
HTTPS protocol.

There exist solutions that help with setting up SSH for git:
git-shell, ssh_acl, and I think most commonly used Gitosis (see
seminal reference about Gitosis on http://git.or.cz/gitwiki/BlogPosts).

> - checkin and checkout using web interface and ssh
> 

Git is distributed version control system: checkin (named 'commit' in
git) and checkout are _local_ operations.  Fetch (getting new changes
from remote repository) and clone (creating new local repository
following or forking specified remote repository) can be done via
local filesystem, via git:// protocol, via "dumb" HTTP, and via SSH.
Push (sending changes to remote repository) needs autheticated
channel: most common used is SSH, but you can also use WebDAV with
HTTPS.

There are web interfaces for Git, something like SVN::Web or ViewVC,
like gitweb (in Perl, developed in git.git repository) and cgit (in C).
See also "Web interfaces" section on InterfacesFrontendsAndTools page
on git wiki.

> - when ever someone checkin something then a email should be send to
>   a email address (a mailing list)

If by 'checkin' you mean publishing changes to a server (i.e. push in
git terminology), see for example contrib/hooks/post-receive-email
hook.
 
> please let me know how to do this with Git

Please learn that Git is _different_ from Subversion, and not try to
follow your SVN workflow and expectations blindly.

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: Few Questions

[Shawn O. Pearce]

Jakub Narebski <jnareb@gmail.com> wrote:
> "Arya, Manish Kumar" <m.arya@yahoo.com> writes:
> > 
> > I want to have following with Git
> > 
> > - LDAP and ssh authentication.
> > 
> 
> Instead of inventing (and failing) its own protocol and its own
> authentication git uses established solutions for authentication: SSH
> for "smart" server, and WebDAV for push via (currently only "dumb")
> HTTPS protocol.
> 
> There exist solutions that help with setting up SSH for git:
> git-shell, ssh_acl, and I think most commonly used Gitosis (see
> seminal reference about Gitosis on http://git.or.cz/gitwiki/BlogPosts).
 
Yup.  Gitosis is very popular for this.

At my last job I rolled my own Gitosis-lite, but in Perl, because
they are a Perl based shop.  It also uses the update-paranoid hook
for access controls, as its more flexible (and easier to confuse
the heck out of yourself) than Gitosis.  But if you can get along
with Gitosis, its a good choice.  Unfortunately its authentication
is limited to public keys registered in the authorized keys file
of the "git" user.

<plug type="not-quite-ready-but-will-be-soon">

Gerrit2 is also likely to enter this "market" soon.  I'm trying to
get it production ready and live for one major project before the
end of this month.  It embeds its own SSH daemon on a non-standard
port, completely disconnected from the OS authentication.  This may
make it slightly more palatable in some enviroments then Gitosis,
as you don't need a huge authorized_keys file, and you don't have
to worry quite so much about attack vectors.

Unlike with Gitosis, public key management is placed on end-users
by using web authentication to identify the user, and letting the
user manage their own "authorized key list".  It also has a full
blown web based code review system built in.  Which right now a
project must use if it is hosted by Gerrit2.  :-)

</plug>

-- 
Shawn.

Re: Few Questions

[R. Tyler Ballance]

On Tue, 2009-01-20 at 08:04 -0800, Shawn O. Pearce wrote:
> <plug type="not-quite-ready-but-will-be-soon">
> 
> Gerrit2 is also likely to enter this "market" soon.  I'm trying to
> get it production ready and live for one major project before the
> end of this month.  It embeds its own SSH daemon on a non-standard
> port, completely disconnected from the OS authentication.  This may
> make it slightly more palatable in some enviroments then Gitosis,
> as you don't need a huge authorized_keys file, and you don't have
> to worry quite so much about attack vectors.

> </plug>
> 

<plug type="not-really-existent-yet-but-gee-golly-i-wish-it-were"
variant="mildly-offtopic" unnecessary:attribute="is-unnecessary">

I talked with Chris Wanstrath (defunkt) of Git Hub once upon a time
about them open sourcing their additions onto the openSSH sshd(8) to
work public key lookup into a MySQL database instead of simply reading a
flat file. 

It'd be nice to add some peer pressure to get them to release it
already ;)

</plug>

I read your thread(s) on groups.google.com regarding Gerrit2 and it
seems quite promising as well. Looking forward to it being ready ;)

Cheers
-- 
-R. Tyler Ballance
Slide, Inc.

Few questions

[Miroslav Vasic]

Hello, everyone!

My name is Miroslav Vasic and I am PhD student on faculty of
Electrical Engineering, University of Politecnica in Madrid. One part of the
project I am working on, as a part of my PhD thesis, is related with
frequency and voltage changing of microprocessors. For that purpose, my
group has bought Viper board with Intel's XScale PXA255 processor. Operating
system which is used for that board is Arcom Embedded Linux, based on Linux
kernel version 2.6. With documentation we got source code of kernel. In
those files some documents which have been written by Mr. Brodowski can be
found, so I believe that people from this mailing list can help me to
clarify some things. I would like to apologize in advance, because I believe
I am asking you some things which are very basic and simple, but my field of
interest is power electronics, not computer programming, so I have lot of
doubts.

 1. In the document user-guide.txt, the second paragraph
"Policy"/"Governor"?, is discussed about solutions about frequency changing.
Later, in other txt files (Governors.txt, core.txt...) I saw that there
is relation between the Governor for my board and the Policy. Furthermore,
 cpufreq_driver is mentioned. I do not understand hierarchy of
these components. As much as I understand now, cpufreq_driver is some kind
of  interface to change CPU clock, Governor is the part which makes decison
when to change clock, and Policy is part where I need to define min and
max frequency, right?

 2. I was trying to understand what was done in the cpufreq.c but I
couldn't. I have lot of doubts how to make a Governor for my system. Do you
have any example where I can see that?

 3. I have tried to control clock with a present interface, from the
userspace, changing the values in some files
(scaling_setspeed, scaling_min/max_frequency). With command cpufreq-info I
saw that it was possible to do this, but I am afraid that this command only
reads values from these files, and that values were not change. Can you
confirm me this?

 4. If I can change clock values through the file scaling_setspeed, then
I need to make a periodic task which will make decision when and how to
change clock values, right? There is no need to write new cpufreq_driver,
Governer or whatever else, isn't it?

 5. If the clock changes are possible through scaling_setspeed file, how
long does it take to change it? I mean, does it change immediately or it
takes time that some periodic process first read new value, and then to
change clock speed?

These are the most important doubts and questions I have right now. Once
again, I am very sorry for disturbing you, but I need a little help and some
explanations. If you don't have time to answer me, please send me some
examples where is cpu frequency control realized, in order to have
better picture about the system I need to make. Right now, I feel little
lost in the great number of new information I need to understand.

 Thanks and best regards,

 Miroslav Vasic

Re: Few questions

[Dominik Brodowski]

Hi,

On Wed, Apr 26, 2006 at 10:04:57AM +0200, Miroslav Vasic wrote:
>  1. In the document user-guide.txt, the second paragraph
> "Policy"/"Governor"?, is discussed about solutions about frequency changing.
> Later, in other txt files (Governors.txt, core.txt...) I saw that there
> is relation between the Governor for my board and the Policy. Furthermore,
>  cpufreq_driver is mentioned. I do not understand hierarchy of
> these components. As much as I understand now, cpufreq_driver is some kind
> of  interface to change CPU clock, Governor is the part which makes decison
> when to change clock, and Policy is part where I need to define min and
> max frequency, right?

Almost... the cpufreq_driver is the hardware driver which follows up on a
request to "set this CPU to somewhere near this frequency". It "talks to the
hardware".

	cpufreq_driver ----> HARDWARE

The governor decides which speed to request. It may do so based upon
userspace feedback, it may always select the highest available frequency, the
lowest one, or it may do so based on "system load" (kstat) as the "ondemand"
or "conservative" governors do.

	governor ----> cpufreq_driver -----> HARDWARE

A "policy" consists of three values: the minimum frequency which is allowed
at the moment, the maximum frequency which is allowed at the moment, and
what governor shall make the decision on what frequency to select.

	policy ----> governor ----> cpufreq_driver -----> HARDWARE

>  2. I was trying to understand what was done in the cpufreq.c but I
> couldn't. I have lot of doubts how to make a Governor for my system. Do you
> have any example where I can see that?

cpufreq.c is sort of the "glue" between all these parts of the cpufreq
subsystem. Regarding a "governor": what do you want to do specifically?
Might one of the existing ones (esp. ondemand or conservative) be good
enough for these purposes? Also, might you want to prefer an userspace-based
approach.

>  3. I have tried to control clock with a present interface, from the
> userspace, changing the values in some files
> (scaling_setspeed, scaling_min/max_frequency). With command cpufreq-info I
> saw that it was possible to do this, but I am afraid that this command only
> reads values from these files, and that values were not change. Can you
> confirm me this?

Yes, exactly. cpufreq-info is for determining information, cpufreq-set is
for setting it. For userspace-based switching of frequency, you should use
the "userspace" governor and "scaling_setspeed". Preferrably, you should use
the libcpufreq library provided for by the cpufrequtils package.

>  4. If I can change clock values through the file scaling_setspeed, then
> I need to make a periodic task which will make decision when and how to
> change clock values, right? There is no need to write new cpufreq_driver,
> Governer or whatever else, isn't it?

If this is what you want to do, you need to
- assert that there is a working cpufreq_driver for your platform (I'm no
  expert in this field on the ARM platform, but if cpufreq-info gives you
  sensible output, it is likely that this driver is provided for.)
- set the policy to some min and max value and the "userspace" governor
- use an userspace daemon:
	- either one which is already implemented (there are tons of them
	  available, some of them broken, some of them not)
	- or write your custom one
  depending on your specific requirements.

>  5. If the clock changes are possible through scaling_setspeed file, how
> long does it take to change it? I mean, does it change immediately or it
> takes time that some periodic process first read new value, and then to
> change clock speed?

Once the write into scaling_setspeed is finalized, the kernel immediately
starts to initiate a clock change. However, due to notifiers which may sleep
(IIRC) or hardware constraints the actual clock change might take some time.
If you need really fast decision-making with regard to CPU frequency scaling,
you should not cross the userspace<->kernel space but make this decision in
kernelspace.

I hope this helped, if not, or if you need more information, just ask :)

	Dominik

Re: Few questions

[Miroslav Vasic]

Thanks for the reply, but I have just one more question (I believe :)) )

>  2. I was trying to understand what was done in the cpufreq.c but I
> > couldn't. I have lot of doubts how to make a Governor for my system. Do
> you
> > have any example where I can see that?
>
> cpufreq.c is sort of the "glue" between all these parts of the cpufreq
> subsystem. Regarding a "governor": what do you want to do specifically?
> Might one of the existing ones (esp. ondemand or conservative) be good
> enough for these purposes? Also, might you want to prefer an
> userspace-based
> approach.


 I would like to see one of those existing governors in order to use them as
my starting point. As I said before, my main concern is a power supply for
this CPU, but still, I need to make something to change the frequency, too.
So, if anyone can send me one of those governors, with corresponding policy
and driver or whatever else, it would be great. As I understood from
previous mail, "ondemand" governor changes frequency depending on the
statistics of the system load. Does it mean that it uses Performance
Monitoring registers of XScale?

Once the write into scaling_setspeed is finalized, the kernel immediately
> starts to initiate a clock change. However, due to notifiers which may
> sleep
> (IIRC) or hardware constraints the actual clock change might take some
> time.
> If you need really fast decision-making with regard to CPU frequency
> scaling,
> you should not cross the userspace<->kernel space but make this decision
> in
> kernelspace.


I plan to make first tests with application which will change frequency
through the scaling_setspeed file, but later I would like to make that
decision in kernelspace, if I manage to understand how to make governor I
need :))

I hope this helped, if not, or if you need more information, just ask :)
>

As you can see I don't hesitate to ask, don't worry :)

Thank you very much, once again.

Miroslav

Re: Few questions

[Dominik Brodowski]

Hi,

On Thu, Apr 27, 2006 at 11:16:03AM +0200, Miroslav Vasic wrote:
> > cpufreq.c is sort of the "glue" between all these parts of the cpufreq
> > subsystem. Regarding a "governor": what do you want to do specifically?
> > Might one of the existing ones (esp. ondemand or conservative) be good
> > enough for these purposes? Also, might you want to prefer an
> > userspace-based
> > approach.
> 
> 
>  I would like to see one of those existing governors in order to use them as
> my starting point.

less /usr/src/linux/drivers/cpufreq/cpufreq_ondemand.c
:)

or, to better understand the calling conventions a simpler one:
less /usr/src/linux/drivers/cpufreq/cpufreq_powersave.c

> As I understood from
> previous mail, "ondemand" governor changes frequency depending on the
> statistics of the system load. Does it mean that it uses Performance
> Monitoring registers of XScale?

No, based on the "kstat" array which is also exported to userspace in
/proc/stat and which is also utilized by top(1), for example.

	Dominik