Re: Reiser4 Upstream Git Repositories on GitHub

[Ivan Shapovalov <intelfx@intelfx.name>]

[-- Attachment #1: Type: text/plain, Size: 10902 bytes --]

On 2016-09-28 at 21:58 +0200, Edward Shishkin wrote:
> 
> On 09/28/2016 05:03 PM, Ivan Shapovalov wrote:
> > On 2016-09-28 at 16:44 +0200, Edward Shishkin wrote:
> > > On 09/28/2016 03:56 PM, Edward Shishkin wrote:
> > > > 
> > > > On 09/28/2016 12:36 PM, Ivan Shapovalov wrote:
> > > > > On 2016-09-28 at 12:17 +0200, Edward Shishkin wrote:
> > > > > > On 09/27/2016 11:51 PM, Ivan Shapovalov wrote:
> > > > > > > On 2016-09-27 at 23:47 +0200, Edward Shishkin wrote:
> > > > > > > > On 09/27/2016 08:36 PM, Ivan Shapovalov wrote:
> > > > > > > > > On 2016-09-27 at 16:13 +0200, Edward Shishkin wrote:
> > > > > > > > > > On 09/27/2016 04:43 AM, Ivan Shapovalov wrote:
> > > > > > > > > > > On 2016-09-27 at 00:37 +0200, Edward Shishkin
> > > > > > > > > > > wrote:
> > > > > > > > > > > > On 09/27/2016 12:05 AM, Ivan Shapovalov wrote:
> > > > > > > > > > > > > On 2016-09-24 at 22:16 +0200, Edward Shishkin
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > Hello everyone,
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > I have set up the updated Namesys
> > > > > > > > > > > > > > repositories
> > > > > > > > > > > > > > at my
> > > > > > > > > > > > > > Github
> > > > > > > > > > > > > > page.
> > > > > > > > > > > > > > Those repositories are supposed to contain
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > latest
> > > > > > > > > > > > > > updates
> > > > > > > > > > > > > > in
> > > > > > > > > > > > > > the (stable) master branch and in other
> > > > > > > > > > > > > > (experimental)
> > > > > > > > > > > > > > branches
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > > I'll announce.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > 1) https://github.com/edward6/reiser4
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > This is a "standalone" reiser4 tree, which
> > > > > > > > > > > > > > doesn't
> > > > > > > > > > > > > > include
> > > > > > > > > > > > > > specific
> > > > > > > > > > > > > > changes of Linux kernel needed for reiser4
> > > > > > > > > > > > > > port. Such
> > > > > > > > > > > > > > changes
> > > > > > > > > > > > > > can
> > > > > > > > > > > > > > be
> > > > > > > > > > > > > > found at the project's page on Sourceforge:
> > > > > > > > > > > > > > https://sourceforge.net/projects/reiser4/
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > An example of work with the standalone
> > > > > > > > > > > > > > reiser4
> > > > > > > > > > > > > > tree:
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > . Patch the respective kernel with the
> > > > > > > > > > > > > > latest
> > > > > > > > > > > > > > available
> > > > > > > > > > > > > > stuff
> > > > > > > > > > > > > > from
> > > > > > > > > > > > > >          Sourceforge;
> > > > > > > > > > > > > > . cd to the "fs" directory;
> > > > > > > > > > > > > > . delete the directory reiser4;
> > > > > > > > > > > > > > . instead of the deleted stuff clone the
> > > > > > > > > > > > > > standalone
> > > > > > > > > > > > > > reiser4
> > > > > > > > > > > > > >          repository from Github;
> > > > > > > > > > > > > > . build and install as usual.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > 2) Libaal and Reiser4progs:
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > https://github.com/edward6/libaal
> > > > > > > > > > > > > > https://github.com/edward6/reiser4progs
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > Before building Libaal and Reiser4progs
> > > > > > > > > > > > > > execute
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > ./prepare
> > > > > > > > > > > > > > script,
> > > > > > > > > > > > > > which will create files needed for build
> > > > > > > > > > > > > > process.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > Edward.
> > > > > > > > > > > > > 
> > > > > > > > > > > > > Wow, finally.
> > > > > > > > > > > > > 
> > > > > > > > > > > > > Maybe we could avoid that "all changes for 10
> > > > > > > > > > > > > years"
> > > > > > > > > > > > > commit?
> > > > > > > > > > > > 
> > > > > > > > > > > > Hi Ivan,
> > > > > > > > > > > > Sorry, don't have a time to granulate it.
> > > > > > > > > > > > 
> > > > > > > > > > > > > I tried to keep track of all patches since
> > > > > > > > > > > > > 3.2...
> > > > > > > > > > > > 
> > > > > > > > > > > > There will be "all changes for 6 years" commit.
> > > > > > > > > > > > Is it much better?
> > > > > > > > > > > 
> > > > > > > > > > > So well, I finished splitting off all known diffs
> > > > > > > > > > > from that
> > > > > > > > > > > big
> > > > > > > > > > > commit.
> > > > > > > > > > > Tt was 12k(+)/8k(-), now it is 7k(+)/7k(-).
> > > > > > > > > > > 
> > > > > > > > > > > The updated branch is here: https://github.com/in
> > > > > > > > > > > telf
> > > > > > > > > > > x/reis
> > > > > > > > > > > er4
> > > > > > > > > > > (unfortunately, not fast-forward).
> > > > > > > > > > > 
> > > > > > > > > > > Moreover, my tree has accumulated quite a few
> > > > > > > > > > > differences
> > > > > > > > > > > from
> > > > > > > > > > > your
> > > > > > > > > > > one. I've dropped trivial discrepancies
> > > > > > > > > > > (comments,
> > > > > > > > > > > formatting
> > > > > > > > > > > etc.)
> > > > > > > > > > > and put the larger ones in separate branches:
> > > > > > > > > > > 
> > > > > > > > > > > 1. https://github.com/intelfx/reiser4/tree/differ
> > > > > > > > > > > ence
> > > > > > > > > > > s/enot
> > > > > > > > > > > ty
> > > > > > > > > > >         (unsupported ioctls return -ENOTTY, not
> > > > > > > > > > > -ENOSYS)
> > > > > > > > > > > 
> > > > > > > > > > > 2. https://github.com/intelfx/reiser4/tree/differ
> > > > > > > > > > > ence
> > > > > > > > > > > s/migr
> > > > > > > > > > > atep
> > > > > > > > > > > age
> > > > > > > > > > >         (the ->migratepage() implementation,
> > > > > > > > > > > which I
> > > > > > > > > > > still do
> > > > > > > > > > > not
> > > > > > > > > > > completely
> > > > > > > > > > >          understand, but it works)
> > > > > > > > > > > 
> > > > > > > > > > > 3. https://github.com/intelfx/reiser4/tree/differ
> > > > > > > > > > > ence
> > > > > > > > > > > s/rena
> > > > > > > > > > > meat
> > > > > > > > > > > 2
> > > > > > > > > > >         (renameat2(RENAME_NOREPLACE)
> > > > > > > > > > > implementation,
> > > > > > > > > > > which
> > > > > > > > > > > you
> > > > > > > > > > > haven't
> > > > > > > > > > >          merged somewhy)
> > > > > > > > > > > 
> > > > > > > > > > > 4. https://github.com/intelfx/reiser4/tree/differ
> > > > > > > > > > > ence
> > > > > > > > > > > s/adju
> > > > > > > > > > > st-t
> > > > > > > > > > > o-3.
> > > > > > > > > > > 15
> > > > > > > > > > >         (part of porting to 3.15 which, again,
> > > > > > > > > > > you
> > > > > > > > > > > haven't
> > > > > > > > > > > merged
> > > > > > > > > > > somewhy)
> > > > > > > > > > > 
> > > > > > > > > > > These branches are on top of that granular
> > > > > > > > > > > "master".
> > > > > > > > > > > Anyway, please take a look.
> > > > > > > > > > 
> > > > > > > > > > It was definitely useful work,
> > > > > > > > > > I'll look at those differences..
> > > > > > > > > 
> > > > > > > > > Maybe you could also consider rebasing things on top
> > > > > > > > > of
> > > > > > > > > that
> > > > > > > > > extracted
> > > > > > > > > granular history?
> > > > > > > > > 
> > > > > > > > 
> > > > > > > > Interesting idea, but I am not able to estimate
> > > > > > > > complexity of such rebasing for now.
> > > > > > > > 
> > > > > > > 
> > > > > > > While we do not have any forks and can afford non-fast-
> > > > > > > forward
> > > > > > > updates
> > > > > > > of master, the complexity is almost nil.
> > > > > > > 
> > > > > > > To rebase your format41 branch, one can do this:
> > > > > > > 
> > > > > > > git rebase --preserve-merges --onto
> > > > > > > 3c7b3c5802e20381496f641fe64b6c1573228c6e
> > > > > > > 8a896fd48ed35c7dfa0188f9b7f4cbdfd469cacb format41
> > > > > > > 
> > > > > > > where 8a896fd is merge-base of format41 and master (that
> > > > > > > big
> > > > > > > commit),
> > > > > > > and 3c7b3c5 is the corresponding commit of the
> > > > > > > synthesized
> > > > > > > history.
> > > > > > > 
> > > > > > > Both commits have identical file content (`git diff
> > > > > > > 8a896fd
> > > > > > > 3c7b3c5`
> > > > > > > yields empty output).
> > > > > > 
> > > > > > OK, everything went smoothly,
> > > > > > Thanks for scrupulously archiving things!
> > > > > > 
> > > > > > Edward.
> > > > > 
> > > > > Great. (In fact, I intended this to be `git push -f`-ed, not
> > > > > `git
> > > > > merge`-ed with original history, but well, git blame now does
> > > > > the
> > > > > right
> > > > > thing, so it's good enough as it is.)
> > > > > 
> > > > > We do not use github pull requests and still send formatted
> > > > > patch
> > > > > series to the ML, correct?
> > > > > 
> > > > 
> > > > Yup, everything to the list, as usual..
> > > 
> > > BTW, your fstrim-scanner is the first candidate to scrub ;)
> > > Actually, I think about a common multi-functional scanner, with 3
> > > modes:
> > > 1) discard only (handle only free blocks);
> > > 2) scrub only (handle only busy blocks);
> > > 3) combined (scan the whole partition; for free blocks call
> > > discard,
> > >       for busy ones call scrub).
> > > Any ideas?
> > > 
> > > Thanks,
> > > Edward.
> > > PS: We have an own ioctl number: 0xCD inherited from
> > > ReiserFS(v3).
> > 
> > I still have to finish the erase unit detection (which has
> > completely
> > stalled) to merge all this work. Moreover:
> > 
> > For the fstrim, we have dropped all locking and serialization
> > issues
> > and declared that fstrim is best-effort: if it misses some blocks
> > due
> > to concurrent transactions allocating and freeing blocks, it
> > doesn't
> > matter.
> > 
> > For the scrub, this won't fly...
> 
> Indeed, the requirements to fstrim and scrub are different,
> but, as I remember, the last decision was to not miss:
> http://marc.info/?l=reiserfs-devel&m=141391883022745&w=2
> so everything will fly just perfectly..
> 
> Edward.

This is different thing, it's about grabbing space in bigger chunks...
If a concurrent transaction allocates some space and frees some space,
we don't care, because it will then be discarded "online".

But in case of the scrub, how do we protect from the storage tree
changing right beneath us?

How is it solved in btrfs? Do they take a temporary snapshot?

-- 
Ivan Shapovalov / intelfx /

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]