What's in ocfs2.git

What's in ocfs2.git

[Joel Becker]

Hey everyone,
	ocfs2.git currently has 85 patches queued for the 2.6.32 merge
window, and that will probably grow to over 100.  So I wanted to fire
off a heads up for anyone who wants to re-review what is in there.
	The big feature item is the reflinkat(2) system call.
reflink(2) was discussed at the filesystem workshop this spring, and the
patch went through various stages of review in May before a final patch
was settled on.  This patch is included here now that ocfs2 provides
support for reflinks.
	All of these patches have been reviewed on the appropriate
mailing lists, some more than once.  So I've decided not to carpet-bomb
with patches.  The git locations are included for those wishing to view
the patches.
	There are two major hunks of the current queued patches.  The
first hunk separates the ocfs2_caching_info structure from ocfs2's
inodes.  We use the caching info to keep track of a metadata buffer's
up2date status in a cluster aware fashion.  Up until now, each buffer
was owned by a single inode, so the cache was tied to inodes.  But now
we need a caching info tied to a non-inode structure that is shared by
many inodes (the refcount tree, see below for more on that).  So we make
it a first-class structure and rework our up2date and journal code to
rely on it.
	Then there is the patch for the reflinkat(2) system call.
	The second major hunk is the implementation of "refcount trees",
the data structure ocfs2 uses to handle reflinks.  Refcount trees allow
for ocfs2 inodes to share data extents in a copy-on-write fashion.
We've been testing the heck out of them, and they're ready to go.

To look at what's currently prepped for the merge window:
[View]
http://git.kernel.org/?p=linux/kernel/git/jlbec/ocfs2.git;a=shortlog;h=merge-window
[Pull]
git://git.kernel.org/pub/scm/linux/kernel/git/jlbec/ocfs2.git merge-window

	The remaining work I have is a cleanup of ocfs2's extended
attribute setter code.  It unifies the code paths for the various ways
ocfs2 stores xattrs.  The current patches have been reviewed on the
mailing list, but I'm still chasing at least one bug.  As such, they're
not in the merge-window branch but are still living on their own xa_loc
branch.  If we get the bugs nailed down before the merge window is over,
we'll be going upstream this cycle.

To look at the extended attribute cleanups:
[View]
http://git.kernel.org/?p=linux/kernel/git/jlbec/ocfs2.git;a=shortlog;h=xa_loc
[Pull]
git://git.kernel.org/pub/scm/linux/kernel/git/jlbec/ocfs2.git xa_loc

Joel


[Merge window shortlog]

Coly Li (1):
      dlmglue.c: add missed mlog lines

Joel Becker (41):
      ocfs2: Make the ocfs2_caching_info structure self-contained.
      ocfs2: Change metadata caching locks to an operations structure.
      ocfs2: Take the inode out of the metadata read/write paths.
      ocfs2: move ip_last_trans to struct ocfs2_caching_info
      ocfs2: move ip_created_trans to struct ocfs2_caching_info
      ocfs2: Pass struct ocfs2_caching_info to the journal functions.
      ocfs2: Store the ocfs2_caching_info on ocfs2_extent_tree.
      ocfs2: Pass ocfs2_caching_info to ocfs2_read_extent_block().
      ocfs2: ocfs2_find_path() only needs the caching info
      ocfs2: ocfs2_create_new_meta_bhs() doesn't need struct inode.
      ocfs2: Pass ocfs2_extent_tree to ocfs2_unlink_path()
      ocfs2: ocfs2_complete_edge_insert() doesn't need struct inode at all.
      ocfs2: Get inode out of ocfs2_rotate_subtree_root_right().
      ocfs2: Pass ocfs2_extent_tree to ocfs2_get_subtree_root()
      ocfs2: Drop struct inode from ocfs2_extent_tree_operations.
      ocfs2: ocfs2_rotate_tree_right() doesn't need struct inode.
      ocfs2: ocfs2_update_edge_lengths() doesn't need struct inode.
      ocfs2: ocfs2_rotate_subtree_left() doesn't need struct inode.
      ocfs2: __ocfs2_rotate_tree_left() doesn't need struct inode.
      ocfs2: ocfs2_rotate_tree_left() no longer needs struct inode.
      ocfs2: ocfs2_merge_rec_left/right() no longer need struct inode.
      ocfs2: ocfs2_try_to_merge_extent() doesn't need struct inode.
      ocfs2: ocfs2_grow_branch() and ocfs2_append_rec_to_path() lose struct inode.
      ocfs2: ocfs2_truncate_rec() doesn't need struct inode.
      ocfs2: Make truncating the extent map an extent_tree_operation.
      ocfs2: ocfs2_insert_at_leaf() doesn't need struct inode.
      ocfs2: Give ocfs2_split_record() an extent_tree instead of an inode.
      ocfs2: ocfs2_do_insert_extent() and ocfs2_insert_path() no longer need an inode.
      ocfs2: ocfs2_extent_contig() only requires the superblock.
      ocfs2: Swap inode for extent_tree in ocfs2_figure_merge_contig_type().
      ocfs2: Remove inode from ocfs2_figure_extent_contig().
      ocfs2: ocfs2_figure_insert_type() no longer needs struct inode.
      ocfs2: Make extent map insertion an extent_tree_operation.
      ocfs2: ocfs2_insert_extent() no longer needs struct inode.
      ocfs2: ocfs2_add_clusters_in_btree() no longer needs struct inode.
      ocfs2: ocfs2_remove_extent() no longer needs struct inode.
      ocfs2: ocfs2_split_and_insert() no longer needs struct inode.
      ocfs2: Teach ocfs2_replace_extent_rec() to use an extent_tree.
      ocfs2: __ocfs2_mark_extent_written() doesn't need struct inode.
      ocfs2: Pass ocfs2_caching_info into ocfs_init_*_extent_tree().
      fs: Add the reflink() operation and reflinkat(2) system call.

Sunil Mushran (1):
      ocfs2: __ocfs2_abort() should not enable panic for local mounts

Tao Ma (41):
      ocfs2: Define refcount tree structure.
      ocfs2: Add metaecc for ocfs2_refcount_block.
      ocfs2: Add ocfs2_read_refcount_block.
      ocfs2: Abstract caching info checkpoint.
      ocfs2: Add new refcount tree lock resource in dlmglue.
      ocfs2: Add caching info for refcount tree.
      ocfs2: Add refcount tree lock mechanism.
      ocfs2: Basic tree root operation.
      ocfs2: Wrap ocfs2_extent_contig in ocfs2_extent_tree.
      ocfs2: Abstract extent split process.
      ocfs2: Add refcount b-tree as a new extent tree.
      ocfs2: move tree path functions to alloc.h.
      ocfs2: Add support for incrementing refcount in the tree.
      ocfs2: Add support of decrementing refcount for delete.
      ocfs2: Add functions for extents refcounted.
      ocfs2: Decrement refcount when truncating refcounted extents.
      ocfs2: Add CoW support.
      ocfs2: CoW refcount tree improvement.
      ocfs2: Integrate CoW in file write.
      ocfs2: CoW a reflinked cluster when it is truncated.
      ocfs2: Add normal functions for reflink a normal file's extents.
      ocfs2: handle file attributes issue for reflink.
      ocfs2: Return extent flags for xattr value tree.
      ocfs2: Abstract duplicate clusters process in CoW.
      ocfs2: Add CoW support for xattr.
      ocfs2: Remove inode from ocfs2_xattr_bucket_get_name_value.
      ocfs2: Abstract the creation of xattr block.
      ocfs2: Abstract ocfs2 xattr tree extend rec iteration process.
      ocfs2: Attach xattr clusters to refcount tree.
      ocfs2: Call refcount tree remove process properly.
      ocfs2: Create an xattr indexed block if needed.
      ocfs2: Add reflink support for xattr.
      ocfs2: Modify removing xattr process for refcount.
      ocfs2: Don't merge in 1st refcount ops of reflink.
      ocfs2: Make transaction extend more efficient.
      ocfs2: Use proper parameter for some inode operation.
      ocfs2: Create reflinked file in orphan dir.
      ocfs2: Add preserve to reflink.
      ocfs2: Implement ocfs2_reflink.
      ocfs2: Enable refcount tree support.
      ocfs2: Add ioctl for reflink.

Wengang Wang (1):
      ocfs2: add spinlock protection when dealing with lockres->purge.


[Extended attribute cleanup shortlog]

Joel Becker (16):
      ocfs2: Introduce ocfs2_xa_loc
      ocfs2: Remove xattrs via ocfs2_xa_loc
      ocfs2: Prefix the member fields of struct ocfs2_xattr_info.
      ocfs2: Add a name_len field to ocfs2_xattr_info.
      ocfs2: Wrap calculation of name+value pair size.
      ocfs2: Set the xattr name+value pair in one place
      ocfs2: Handle value tree roots in ocfs2_xa_set_inline_value()
      ocfs2: Provide ocfs2_xa_fill_value_buf() for external value processing
      ocfs2: Teach ocfs2_xa_loc how to do its own journal work
      ocfs2: Allocation in ocfs2_xa_prepare_entry(), values in ocfs2_xa_store_value()
      ocfs2: Gell into ocfs2_xa_set()
      ocfs2: Let ocfs2_xa_prepare_entry() do space checks.
      ocfs2: Set xattr block entries with ocfs2_xa_set()
      ocfs2: Set inline xattr entries with ocfs2_xa_set()
      ocfs2: Handle errors while setting external xattr values.
      ocfs2: Never call ocfs2_xa_fill_value_buf() for inline values.


-- 

"I inject pure kryptonite into my brain.
 It improves my kung fu, and it eases the pain."


Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: What's in ocfs2.git

[Mark Fasheh]

Hi Steve,

On Mon, Dec 04, 2006 at 10:54:53AM +0000, Steven Whitehouse wrote:
> > In the future, I'd like to see a "relative atime" mode, which functions
> > in the manner described by Valerie Henson at:
> > 
> > http://lkml.org/lkml/2006/8/25/380
> > 
> I'd like to second that. [adding Val Henson to the "to"] What (if
> anything) remains to be done before the relative atime patch is ready to
> go upstream? I'm happy to help out here if required,
Last time I looked at them, things seemed to be in pretty good shape - it
wasn't a very large patch series.

The thing is (I'm going from memory here), gfs2 and ocfs2 are likely to just
make use of the option parsing (and setting of the MNT_RELATIME flag), and
ignore the changes to touch_atime() since we we handle our own atime
updates.

Overall I think it's a matter of pushing the patches to the kernel and to
mount(8). For ocfs2/gfs2 we implement a small amount of the logic in our
"lock and update atime" functions.
	--Mark

--
Mark Fasheh
Senior Software Developer, Oracle
mark.fasheh@oracle.com

Re: What's in ocfs2.git

[Steven Whitehouse]

Hi,

On Sun, 2006-12-03 at 12:31 -0800, Mark Fasheh wrote:
> This e-mail describes the OCFS2 patches which I intend to push
> upstream to Linus for 2.6.20.
> 
> * Atime updates - thanks to Tiger Yang <tiger.yang@oracle.com>, ocfs2 now
>   writes to the inode atime field. This doesn't require any disk changes,
>   and is completely backwards compatible with older ocfs2 versions. An
>   inodes Atime is only updated if it hasn't changed within a certain
>   quantum. The user can define their own value at mount time, with 0
>   indicating that atime should always be updated. This is very similar to
>   the scheme implemented by gfs2. In the future, I'd like to see a "relative
>   atime" mode, which functions in the manner described by Valerie Henson at:
> 
> http://lkml.org/lkml/2006/8/25/380
> 
I'd like to second that. [adding Val Henson to the "to"] What (if
anything) remains to be done before the relative atime patch is ready to
go upstream? I'm happy to help out here if required,

Steve.

What's in ocfs2.git

[Mark Fasheh]

This e-mail describes the OCFS2 patches which I intend to push
upstream to Linus for 2.6.20.

* Various ocfs2 cleanups, including a patchset by me intended to clean up
  some of the internal ocfs2 journal api. Mostly this revolves around
  removing the ocfs2_journal_handle wrapper around handle_t. The immediate
  benefits are better readability and a slightly smaller memory footprint
  for open journal transactions.


* Configfs gets some small cleanups and some mutex annotations.


* Atime updates - thanks to Tiger Yang <tiger.yang@oracle.com>, ocfs2 now
  writes to the inode atime field. This doesn't require any disk changes,
  and is completely backwards compatible with older ocfs2 versions. An
  inodes Atime is only updated if it hasn't changed within a certain
  quantum. The user can define their own value at mount time, with 0
  indicating that atime should always be updated. This is very similar to
  the scheme implemented by gfs2. In the future, I'd like to see a "relative
  atime" mode, which functions in the manner described by Valerie Henson at:

http://lkml.org/lkml/2006/8/25/380


* sys_splice - ocfs2 now has splice read and write support. Thanks again to
  Tiger for the bulk of this functionality. Mostly we make use of the
  generic_splice_read() and generic_file_splice_write_nolock() functions
  provided already in fs/splice.c.

 - There is one patch in the ocfs2 splice() series external to fs/ocfs2 - a
   simple export of should_remove_suid(). This is done for code reuse
   purposes. That particular patch can be seen at:

http://ftp.kernel.org/pub/linux/kernel/people/mfasheh/ocfs2/ocfs2_git_patches/ocfs2-upstream-linus-20061201/0025-Export-should_remove_suid.txt

   I'll also attach it to this mail for review purposes.


* Last in the list of notable patches is a somewhat involved fix by Kurt
  Hackel <kurt.hackel@oracle.com> within the ocfs2 dlm. We had temporarily
  disable automatic migration of certain lock types because it was causing
  us problems during stress testing. This patch fixes those problems and
  re-enables migration. Overall this should reduce memory usage of the ocfs2
  dlm.


The patches can be found in

git://git.kernel.org/pub/scm/linux/kernel/git/mfasheh/ocfs2.git upstream-linus

Additionally, broken out patches are available at:

http://ftp.kernel.org/pub/linux/kernel/people/mfasheh/ocfs2/ocfs2_git_patches/ocfs2-upstream-linus-20060924/

Thanks,
	--Mark

--
Mark Fasheh
Senior Software Developer, Oracle
mark.fasheh@oracle.com


[PATCH] Export should_remove_suid()

This helps us avoid replicating the same logic within file system drivers.

Signed-off-by: Mark Fasheh <mark.fasheh@oracle.com>

---

 mm/filemap.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

d23a147bb6e8d467e8df73b6589888717da3b9ce
diff --git a/mm/filemap.c b/mm/filemap.c
index 7b84dc8..13df01c 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -1893,6 +1893,7 @@ int should_remove_suid(struct dentry *de
 
 	return 0;
 }
+EXPORT_SYMBOL(should_remove_suid);
 
 int __remove_suid(struct dentry *dentry, int kill)
 {
-- 
1.3.3

What's in ocfs2.git

[Mark Fasheh]

The following describes the contents of the ALL branch of

git://git.kernel.org/pub/scm/linux/kernel/git/mfasheh/ocfs2.git

* The usual set of cleanups and fixes.
   - a #include cleanup
   - some nlink check fixes
   - a heartbeat sector math fix

* OCFS2 now supports ext2 file attributes. Thanks to Herbert Poetzl for that
  feature.

* Shared writeable mmap support. This just make use of the ->page_mkwrite()
  callback to take write locks on pages. It works quite well, though there's
  one unresolved issue I'd like to work out first.

* Fix mtime updates on buffered writes so that they always happen.

* An old directory readahead patch which I had never pushed upstream. IMHO,
  it's actually more interesting in that it provides the code to do readahead
  of other fs structures in the future.

* A patch series which removes the "dentry" vote, in favor of a locking
  scheme which achieves the same thing (cluster-wide dentry delete on
  unlink/rename) using fewer network messages in the average case. This
  series also includes two interesting changes:
   - An OCFS2 DLM API update, needed to get this going.
   - A small patch to the VFS to enable a file system to manually d_move()
     during rename. This also updates NFS for the behavior.

* A configfs fix - it was previously possible to create duplicate subsystem
  names.


Most of this will be queued up for 2.6.19. Diffstat and shortlog are below.  

Broken out patches can be found at:

http://www.kernel.org/pub/linux/kernel/people/mfasheh/ocfs2/ocfs2_git_patches/ocfs2-all-20060913/
	--Mark

 fs/configfs/dir.c               |   32 ++
 fs/namei.c                      |    6 
 fs/nfs/dir.c                    |    3 
 fs/nfs/super.c                  |   10 
 fs/ocfs2/Makefile               |    1 
 fs/ocfs2/alloc.c                |   28 +-
 fs/ocfs2/aops.c                 |   83 ++----
 fs/ocfs2/buffer_head_io.c       |   95 +++++--
 fs/ocfs2/buffer_head_io.h       |    2 
 fs/ocfs2/cluster/heartbeat.c    |    8 
 fs/ocfs2/cluster/tcp_internal.h |    5 
 fs/ocfs2/dcache.c               |  380 +++++++++++++++++++++++++++++-
 fs/ocfs2/dcache.h               |   27 ++
 fs/ocfs2/dir.c                  |   28 +-
 fs/ocfs2/dlm/dlmapi.h           |    1 
 fs/ocfs2/dlm/dlmast.c           |    6 
 fs/ocfs2/dlm/dlmcommon.h        |    1 
 fs/ocfs2/dlm/dlmlock.c          |   10 
 fs/ocfs2/dlm/dlmmaster.c        |    4 
 fs/ocfs2/dlm/dlmrecovery.c      |    3 
 fs/ocfs2/dlm/userdlm.c          |   81 ++----
 fs/ocfs2/dlm/userdlm.h          |    1 
 fs/ocfs2/dlmglue.c              |  497 +++++++++++++++++++++++++++++++---------
 fs/ocfs2/dlmglue.h              |   16 +
 fs/ocfs2/export.c               |    4 
 fs/ocfs2/file.c                 |    3 
 fs/ocfs2/inode.c                |   42 ++-
 fs/ocfs2/inode.h                |    3 
 fs/ocfs2/ioctl.c                |  136 ++++++++++
 fs/ocfs2/ioctl.h                |   16 +
 fs/ocfs2/mmap.c                 |  100 ++++++--
 fs/ocfs2/namei.c                |  148 +++++++----
 fs/ocfs2/ocfs2_fs.h             |   24 +
 fs/ocfs2/ocfs2_lockid.h         |   25 ++
 fs/ocfs2/super.c                |    2 
 fs/ocfs2/sysfile.c              |    4 
 fs/ocfs2/uptodate.c             |   21 +
 fs/ocfs2/uptodate.h             |    2 
 fs/ocfs2/vote.c                 |  180 --------------
 fs/ocfs2/vote.h                 |    5 
 include/linux/fs.h              |    7 
 41 files changed, 1479 insertions(+), 571 deletions(-)

Adrian Bunk:
      fs/ocfs2/ioctl.c should #include "ioctl.h"

Herbert Poetzl:
      ocfs2: add ext2 attributes

Joel Becker:
      configfs: Prevent duplicate subsystem names.

Mark Fasheh:
      ocfs2: implement directory read-ahead
      ocfs2: Shared writeable mmap
      ocfs2: properly update i_mtime on buffered write
      ocfs2: move nlink check in ocfs2_mknod()
      ocfs2: Remove overzealous BUG_ON()
      ocfs2: Silence dlm error print
      ocfs2: Allow binary names in the DLM
      ocfs2: Update dlmfs for new dlmlock() API
      ocfs2: Update dlmglue for new dlmlock() API
      ocfs2: Add new cluster lock type
      ocfs2: Add dentry tracking API
      ocfs2: Hook rest of the file system into dentry locking API
      ocfs2: Remove the dentry vote
      Allow file systems to manually d_move() inside of ->rename()
      ocfs2: manually d_move() during ocfs2_rename()

Mathieu Avila:
      ocfs2: Fix heartbeat sector calculation

Tiger Yang:
      ocfs2: Fix directory link count checks in ocfs2_link()