[PATCH] doc/rcu: Correct field_count field naming in examples

[PATCH] doc/rcu: Correct field_count field naming in examples

[Joel Fernandes (Google)]

I believe this field should be called field_count instead of file_count.
Correct the doc with the same.

Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
 Documentation/RCU/listRCU.txt | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/RCU/listRCU.txt b/Documentation/RCU/listRCU.txt
index adb5a3782846..190e666fc359 100644
--- a/Documentation/RCU/listRCU.txt
+++ b/Documentation/RCU/listRCU.txt
@@ -175,7 +175,7 @@ otherwise, the added fields would need to be filled in):
 		list_for_each_entry(e, list, list) {
 			if (!audit_compare_rule(rule, &e->rule)) {
 				e->rule.action = newaction;
-				e->rule.file_count = newfield_count;
+				e->rule.field_count = newfield_count;
 				write_unlock(&auditsc_lock);
 				return 0;
 			}
@@ -204,7 +204,7 @@ RCU ("read-copy update") its name.  The RCU code is as follows:
 					return -ENOMEM;
 				audit_copy_rule(&ne->rule, &e->rule);
 				ne->rule.action = newaction;
-				ne->rule.file_count = newfield_count;
+				ne->rule.field_count = newfield_count;
 				list_replace_rcu(&e->list, &ne->list);
 				call_rcu(&e->rcu, audit_free_rule);
 				return 0;
-- 
2.21.0.1020.gf2820cf01a-goog

Re: [PATCH] doc/rcu: Correct field_count field naming in examples

[Paul E. McKenney]

On Sat, May 04, 2019 at 10:03:10PM -0400, Joel Fernandes (Google) wrote:
> I believe this field should be called field_count instead of file_count.
> Correct the doc with the same.
> 
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>

But if we are going to update this, why not update it with the current
audit_filter_task(), audit_del_rule(), and audit_add_rule() code?

Hmmm...  One reason is that some of them have changed beyond recognition.

And this example code predates v2.6.12.  ;-)

So good eyes, but I believe that this really does reflect the ancient
code...

On the other hand, would you have ideas for more modern replacement
examples?

							Thanx, Paul

> ---
>  Documentation/RCU/listRCU.txt | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/RCU/listRCU.txt b/Documentation/RCU/listRCU.txt
> index adb5a3782846..190e666fc359 100644
> --- a/Documentation/RCU/listRCU.txt
> +++ b/Documentation/RCU/listRCU.txt
> @@ -175,7 +175,7 @@ otherwise, the added fields would need to be filled in):
>  		list_for_each_entry(e, list, list) {
>  			if (!audit_compare_rule(rule, &e->rule)) {
>  				e->rule.action = newaction;
> -				e->rule.file_count = newfield_count;
> +				e->rule.field_count = newfield_count;
>  				write_unlock(&auditsc_lock);
>  				return 0;
>  			}
> @@ -204,7 +204,7 @@ RCU ("read-copy update") its name.  The RCU code is as follows:
>  					return -ENOMEM;
>  				audit_copy_rule(&ne->rule, &e->rule);
>  				ne->rule.action = newaction;
> -				ne->rule.file_count = newfield_count;
> +				ne->rule.field_count = newfield_count;
>  				list_replace_rcu(&e->list, &ne->list);
>  				call_rcu(&e->rcu, audit_free_rule);
>  				return 0;
> -- 
> 2.21.0.1020.gf2820cf01a-goog
> 

Re: [PATCH] doc/rcu: Correct field_count field naming in examples

[Joel Fernandes]

On Mon, May 06, 2019 at 05:04:53PM -0700, Paul E. McKenney wrote:
> On Sat, May 04, 2019 at 10:03:10PM -0400, Joel Fernandes (Google) wrote:
> > I believe this field should be called field_count instead of file_count.
> > Correct the doc with the same.
> > 
> > Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> 
> But if we are going to update this, why not update it with the current
> audit_filter_task(), audit_del_rule(), and audit_add_rule() code?
> 
> Hmmm...  One reason is that some of them have changed beyond recognition.

It seems to me that these 3 functions are just structured differently but is
conceptually the same.

There is now an array of lists stored in audit_filter_list. Each list is a
set of rules. Versus in the listRCU.txt, there is only one global.

The other difference is there is a mutex held &audit_filter_mutex
audit_{add,del}_rule. Where as in listRCU, it says that is not needed since
another mutex is already held.

> And this example code predates v2.6.12.  ;-)
> 
> So good eyes, but I believe that this really does reflect the ancient
> code...
> 
> On the other hand, would you have ideas for more modern replacement
> examples?

There are 3 cases I can see in listRCU.txt:
  (1) action taken outside of read_lock (can tolerate stale data), no in-place update.
                this is the best possible usage of RCU.
  (2) action taken outside of read_lock, in-place updates
                this is good as long as not too many in-place updates.
                involves copying creating new list node and replacing the
                node being updated with it.
  (3) cannot tolerate stale data: here a deleted or obsolete flag can be used
                                  protected by a per-entry lock. reader
				  aborts if object is stale.

Any replacement example must make satisfy (3) too?

The only example for (3) that I know of is sysvipc sempahores which you also
mentioned in the paper. Looking through this code, it hasn't changed
conceptually and it could be a fit for an example (ipc_valid_object() checks
for whether the object is stale).

The other example could be dentry look up which uses seqlocks for the
RCU-walk case? But that could be too complex. This is also something I first
learnt from the paper and then the excellent path-lookup.rst document in
kernel sources.

I will keep any eye out for other examples in the kernel code as well.

Let me know what you think, thanks!

 - Joel


> > ---
> >  Documentation/RCU/listRCU.txt | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/Documentation/RCU/listRCU.txt b/Documentation/RCU/listRCU.txt
> > index adb5a3782846..190e666fc359 100644
> > --- a/Documentation/RCU/listRCU.txt
> > +++ b/Documentation/RCU/listRCU.txt
> > @@ -175,7 +175,7 @@ otherwise, the added fields would need to be filled in):
> >  		list_for_each_entry(e, list, list) {
> >  			if (!audit_compare_rule(rule, &e->rule)) {
> >  				e->rule.action = newaction;
> > -				e->rule.file_count = newfield_count;
> > +				e->rule.field_count = newfield_count;
> >  				write_unlock(&auditsc_lock);
> >  				return 0;
> >  			}
> > @@ -204,7 +204,7 @@ RCU ("read-copy update") its name.  The RCU code is as follows:
> >  					return -ENOMEM;
> >  				audit_copy_rule(&ne->rule, &e->rule);
> >  				ne->rule.action = newaction;
> > -				ne->rule.file_count = newfield_count;
> > +				ne->rule.field_count = newfield_count;
> >  				list_replace_rcu(&e->list, &ne->list);
> >  				call_rcu(&e->rcu, audit_free_rule);
> >  				return 0;
> > -- 
> > 2.21.0.1020.gf2820cf01a-goog
> > 
> 

Re: [PATCH] doc/rcu: Correct field_count field naming in examples

[Paul E. McKenney]

On Wed, May 08, 2019 at 12:26:35PM -0400, Joel Fernandes wrote:
> On Mon, May 06, 2019 at 05:04:53PM -0700, Paul E. McKenney wrote:
> > On Sat, May 04, 2019 at 10:03:10PM -0400, Joel Fernandes (Google) wrote:
> > > I believe this field should be called field_count instead of file_count.
> > > Correct the doc with the same.
> > > 
> > > Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> > 
> > But if we are going to update this, why not update it with the current
> > audit_filter_task(), audit_del_rule(), and audit_add_rule() code?
> > 
> > Hmmm...  One reason is that some of them have changed beyond recognition.
> 
> It seems to me that these 3 functions are just structured differently but is
> conceptually the same.
> 
> There is now an array of lists stored in audit_filter_list. Each list is a
> set of rules. Versus in the listRCU.txt, there is only one global.
> 
> The other difference is there is a mutex held &audit_filter_mutex
> audit_{add,del}_rule. Where as in listRCU, it says that is not needed since
> another mutex is already held.

Agreed.

> > And this example code predates v2.6.12.  ;-)
> > 
> > So good eyes, but I believe that this really does reflect the ancient
> > code...
> > 
> > On the other hand, would you have ideas for more modern replacement
> > examples?
> 
> There are 3 cases I can see in listRCU.txt:
>   (1) action taken outside of read_lock (can tolerate stale data), no in-place update.
>                 this is the best possible usage of RCU.
>   (2) action taken outside of read_lock, in-place updates
>                 this is good as long as not too many in-place updates.
>                 involves copying creating new list node and replacing the
>                 node being updated with it.
>   (3) cannot tolerate stale data: here a deleted or obsolete flag can be used
>                                   protected by a per-entry lock. reader
> 				  aborts if object is stale.
> 
> Any replacement example must make satisfy (3) too?

It would be OK to have a separate example for (3).  It would of course
be nicer to have one example for all three, but not all -that- important.

> The only example for (3) that I know of is sysvipc sempahores which you also
> mentioned in the paper. Looking through this code, it hasn't changed
> conceptually and it could be a fit for an example (ipc_valid_object() checks
> for whether the object is stale).

That is indeed the classic canonical example.  ;-)

> The other example could be dentry look up which uses seqlocks for the
> RCU-walk case? But that could be too complex. This is also something I first
> learnt from the paper and then the excellent path-lookup.rst document in
> kernel sources.

This is a great example, but it would need serious simplification for
use in the Documentation/RCU directory.  Note that dcache uses it to
gain very limited and targeted consistency -- only a few types of updates
acquire the write-side of that seqlock.

Might be quite worthwhile to have a simplified example, though!
Perhaps a trivial hash table where write-side sequence lock is acquired
only when moving an element from one chain to another?

> I will keep any eye out for other examples in the kernel code as well.

Very good!

							Thanx, Paul

> Let me know what you think, thanks!
> 
>  - Joel
> 
> 
> > > ---
> > >  Documentation/RCU/listRCU.txt | 4 ++--
> > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/Documentation/RCU/listRCU.txt b/Documentation/RCU/listRCU.txt
> > > index adb5a3782846..190e666fc359 100644
> > > --- a/Documentation/RCU/listRCU.txt
> > > +++ b/Documentation/RCU/listRCU.txt
> > > @@ -175,7 +175,7 @@ otherwise, the added fields would need to be filled in):
> > >  		list_for_each_entry(e, list, list) {
> > >  			if (!audit_compare_rule(rule, &e->rule)) {
> > >  				e->rule.action = newaction;
> > > -				e->rule.file_count = newfield_count;
> > > +				e->rule.field_count = newfield_count;
> > >  				write_unlock(&auditsc_lock);
> > >  				return 0;
> > >  			}
> > > @@ -204,7 +204,7 @@ RCU ("read-copy update") its name.  The RCU code is as follows:
> > >  					return -ENOMEM;
> > >  				audit_copy_rule(&ne->rule, &e->rule);
> > >  				ne->rule.action = newaction;
> > > -				ne->rule.file_count = newfield_count;
> > > +				ne->rule.field_count = newfield_count;
> > >  				list_replace_rcu(&e->list, &ne->list);
> > >  				call_rcu(&e->rcu, audit_free_rule);
> > >  				return 0;
> > > -- 
> > > 2.21.0.1020.gf2820cf01a-goog
> > > 
> > 
> 

Re: [PATCH] doc/rcu: Correct field_count field naming in examples

[Andrea Parri]

Hi Paul, Joel,

> > > On the other hand, would you have ideas for more modern replacement
> > > examples?
> > 
> > There are 3 cases I can see in listRCU.txt:
> >   (1) action taken outside of read_lock (can tolerate stale data), no in-place update.
> >                 this is the best possible usage of RCU.
> >   (2) action taken outside of read_lock, in-place updates
> >                 this is good as long as not too many in-place updates.
> >                 involves copying creating new list node and replacing the
> >                 node being updated with it.
> >   (3) cannot tolerate stale data: here a deleted or obsolete flag can be used
> >                                   protected by a per-entry lock. reader
> > 				  aborts if object is stale.
> > 
> > Any replacement example must make satisfy (3) too?
> 
> It would be OK to have a separate example for (3).  It would of course
> be nicer to have one example for all three, but not all -that- important.
> 
> > The only example for (3) that I know of is sysvipc sempahores which you also
> > mentioned in the paper. Looking through this code, it hasn't changed
> > conceptually and it could be a fit for an example (ipc_valid_object() checks
> > for whether the object is stale).
> 
> That is indeed the classic canonical example.  ;-)
> 
> > The other example could be dentry look up which uses seqlocks for the
> > RCU-walk case? But that could be too complex. This is also something I first
> > learnt from the paper and then the excellent path-lookup.rst document in
> > kernel sources.
> 
> This is a great example, but it would need serious simplification for
> use in the Documentation/RCU directory.  Note that dcache uses it to
> gain very limited and targeted consistency -- only a few types of updates
> acquire the write-side of that seqlock.
> 
> Might be quite worthwhile to have a simplified example, though!
> Perhaps a trivial hash table where write-side sequence lock is acquired
> only when moving an element from one chain to another?

Sorry to take you down here..., but what do you mean by "the paper"?  ;-/

Thanx,
  Andrea

Re: [PATCH] doc/rcu: Correct field_count field naming in examples

[Paul E. McKenney]

On Sun, May 12, 2019 at 12:11:26AM +0200, Andrea Parri wrote:
> Hi Paul, Joel,
> 
> > > > On the other hand, would you have ideas for more modern replacement
> > > > examples?
> > > 
> > > There are 3 cases I can see in listRCU.txt:
> > >   (1) action taken outside of read_lock (can tolerate stale data), no in-place update.
> > >                 this is the best possible usage of RCU.
> > >   (2) action taken outside of read_lock, in-place updates
> > >                 this is good as long as not too many in-place updates.
> > >                 involves copying creating new list node and replacing the
> > >                 node being updated with it.
> > >   (3) cannot tolerate stale data: here a deleted or obsolete flag can be used
> > >                                   protected by a per-entry lock. reader
> > > 				  aborts if object is stale.
> > > 
> > > Any replacement example must make satisfy (3) too?
> > 
> > It would be OK to have a separate example for (3).  It would of course
> > be nicer to have one example for all three, but not all -that- important.
> > 
> > > The only example for (3) that I know of is sysvipc sempahores which you also
> > > mentioned in the paper. Looking through this code, it hasn't changed
> > > conceptually and it could be a fit for an example (ipc_valid_object() checks
> > > for whether the object is stale).
> > 
> > That is indeed the classic canonical example.  ;-)
> > 
> > > The other example could be dentry look up which uses seqlocks for the
> > > RCU-walk case? But that could be too complex. This is also something I first
> > > learnt from the paper and then the excellent path-lookup.rst document in
> > > kernel sources.
> > 
> > This is a great example, but it would need serious simplification for
> > use in the Documentation/RCU directory.  Note that dcache uses it to
> > gain very limited and targeted consistency -- only a few types of updates
> > acquire the write-side of that seqlock.
> > 
> > Might be quite worthwhile to have a simplified example, though!
> > Perhaps a trivial hash table where write-side sequence lock is acquired
> > only when moving an element from one chain to another?
> 
> Sorry to take you down here..., but what do you mean by "the paper"?  ;-/

One or both of these two:

http://www2.rdrop.com/~paulmck/techreports/survey.2012.09.17a.pdf
http://www2.rdrop.com/~paulmck/techreports/RCUUsage.2013.02.24a.pdf

							Thanx, Paul

Re: [PATCH] doc/rcu: Correct field_count field naming in examples

[Andrea Parri]

On Sat, May 11, 2019 at 05:41:31PM -0700, Paul E. McKenney wrote:
> On Sun, May 12, 2019 at 12:11:26AM +0200, Andrea Parri wrote:
> > Hi Paul, Joel,
> > 
> > > > > On the other hand, would you have ideas for more modern replacement
> > > > > examples?
> > > > 
> > > > There are 3 cases I can see in listRCU.txt:
> > > >   (1) action taken outside of read_lock (can tolerate stale data), no in-place update.
> > > >                 this is the best possible usage of RCU.
> > > >   (2) action taken outside of read_lock, in-place updates
> > > >                 this is good as long as not too many in-place updates.
> > > >                 involves copying creating new list node and replacing the
> > > >                 node being updated with it.
> > > >   (3) cannot tolerate stale data: here a deleted or obsolete flag can be used
> > > >                                   protected by a per-entry lock. reader
> > > > 				  aborts if object is stale.
> > > > 
> > > > Any replacement example must make satisfy (3) too?
> > > 
> > > It would be OK to have a separate example for (3).  It would of course
> > > be nicer to have one example for all three, but not all -that- important.
> > > 
> > > > The only example for (3) that I know of is sysvipc sempahores which you also
> > > > mentioned in the paper. Looking through this code, it hasn't changed
> > > > conceptually and it could be a fit for an example (ipc_valid_object() checks
> > > > for whether the object is stale).
> > > 
> > > That is indeed the classic canonical example.  ;-)
> > > 
> > > > The other example could be dentry look up which uses seqlocks for the
> > > > RCU-walk case? But that could be too complex. This is also something I first
> > > > learnt from the paper and then the excellent path-lookup.rst document in
> > > > kernel sources.
> > > 
> > > This is a great example, but it would need serious simplification for
> > > use in the Documentation/RCU directory.  Note that dcache uses it to
> > > gain very limited and targeted consistency -- only a few types of updates
> > > acquire the write-side of that seqlock.
> > > 
> > > Might be quite worthwhile to have a simplified example, though!
> > > Perhaps a trivial hash table where write-side sequence lock is acquired
> > > only when moving an element from one chain to another?
> > 
> > Sorry to take you down here..., but what do you mean by "the paper"?  ;-/
> 
> One or both of these two:
> 
> http://www2.rdrop.com/~paulmck/techreports/survey.2012.09.17a.pdf
> http://www2.rdrop.com/~paulmck/techreports/RCUUsage.2013.02.24a.pdf

Oh, these are familiar.  ;-) Thank you!

  Andrea

Re: [PATCH] doc/rcu: Correct field_count field naming in examples

[Joel Fernandes]

On Wed, May 08, 2019 at 11:16:38AM -0700, Paul E. McKenney wrote:
[snip]
> > The other example could be dentry look up which uses seqlocks for the
> > RCU-walk case? But that could be too complex. This is also something I first
> > learnt from the paper and then the excellent path-lookup.rst document in
> > kernel sources.
> 
> This is a great example, but it would need serious simplification for
> use in the Documentation/RCU directory.  Note that dcache uses it to
> gain very limited and targeted consistency -- only a few types of updates
> acquire the write-side of that seqlock.
> 
> Might be quite worthwhile to have a simplified example, though!
> Perhaps a trivial hash table where write-side sequence lock is acquired
> only when moving an element from one chain to another?

Here you meant "moving from one chain to another" in the case of
hashtable-resizing right? I could not think of another reason why an element
is moved between 2 hash chains.

I just finished reading the main parts of Josh's relativistic hashtable paper
[1] and it is very cool indeed. The whole wait-for-readers application for
hashtable expansion is so well thought. I am planning to go over more papers
and code and can certainly update this example with a read-mostly hashtable
example as well as you are suggesting. :-)

[1] https://www.usenix.org/legacy/event/atc11/tech/final_files/Triplett.pdf

thanks,

 - Joel

Re: [PATCH] doc/rcu: Correct field_count field naming in examples

[Paul E. McKenney]

On Sun, May 12, 2019 at 11:43:05PM -0400, Joel Fernandes wrote:
> On Wed, May 08, 2019 at 11:16:38AM -0700, Paul E. McKenney wrote:
> [snip]
> > > The other example could be dentry look up which uses seqlocks for the
> > > RCU-walk case? But that could be too complex. This is also something I first
> > > learnt from the paper and then the excellent path-lookup.rst document in
> > > kernel sources.
> > 
> > This is a great example, but it would need serious simplification for
> > use in the Documentation/RCU directory.  Note that dcache uses it to
> > gain very limited and targeted consistency -- only a few types of updates
> > acquire the write-side of that seqlock.
> > 
> > Might be quite worthwhile to have a simplified example, though!
> > Perhaps a trivial hash table where write-side sequence lock is acquired
> > only when moving an element from one chain to another?
> 
> Here you meant "moving from one chain to another" in the case of
> hashtable-resizing right? I could not think of another reason why an element
> is moved between 2 hash chains.

Either that or in terms of atomic rekeying of a specific element in that
table, thus potentially requiring an atomic move of only that specific
element to another hash chain.

> I just finished reading the main parts of Josh's relativistic hashtable paper
> [1] and it is very cool indeed. The whole wait-for-readers application for
> hashtable expansion is so well thought. I am planning to go over more papers
> and code and can certainly update this example with a read-mostly hashtable
> example as well as you are suggesting. :-)
> 
> [1] https://www.usenix.org/legacy/event/atc11/tech/final_files/Triplett.pdf

Sounds very good!

							Thanx, Paul

Re: [PATCH] doc/rcu: Correct field_count field naming in examples

[Joel Fernandes]

On Wed, May 8, 2019 at 9:16 PM Paul E. McKenney <paulmck@linux.ibm.com> wrote:
[snip]
> > > And this example code predates v2.6.12.  ;-)
> > >
> > > So good eyes, but I believe that this really does reflect the ancient
> > > code...
> > >
> > > On the other hand, would you have ideas for more modern replacement
> > > examples?
> >
> > There are 3 cases I can see in listRCU.txt:
> >   (1) action taken outside of read_lock (can tolerate stale data), no in-place update.
> >                 this is the best possible usage of RCU.
> >   (2) action taken outside of read_lock, in-place updates
> >                 this is good as long as not too many in-place updates.
> >                 involves copying creating new list node and replacing the
> >                 node being updated with it.
> >   (3) cannot tolerate stale data: here a deleted or obsolete flag can be used
> >                                   protected by a per-entry lock. reader
> >                                 aborts if object is stale.
> >
> > Any replacement example must make satisfy (3) too?
>
> It would be OK to have a separate example for (3).  It would of course
> be nicer to have one example for all three, but not all -that- important.
>
> > The only example for (3) that I know of is sysvipc sempahores which you also
> > mentioned in the paper. Looking through this code, it hasn't changed
> > conceptually and it could be a fit for an example (ipc_valid_object() checks
> > for whether the object is stale).
>
> That is indeed the classic canonical example.  ;-)

FWIW just want to mention, it seems to me the ptrace task list
traversal could be a great example of "mark obsolete objects" and is
simple so I could just use that.
Neil talks about it in his article here:
https://lwn.net/Articles/610972/ . In "Group 3: Transform the way the
list is walked"

Cheers,
- Joel