LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Evgeniy Polyakov <johnpol@2ka.mipt.ru>
To: Eric Dumazet <dada1@cosmosbay.com>
Cc: Johann Borck <johann.borck@densedata.com>,
	Ulrich Drepper <drepper@redhat.com>,
	Ulrich Drepper <drepper@gmail.com>,
	lkml <linux-kernel@vger.kernel.org>,
	David Miller <davem@davemloft.net>, Andrew Morton <akpm@osdl.org>,
	netdev <netdev@vger.kernel.org>,
	Zach Brown <zach.brown@oracle.com>,
	Christoph Hellwig <hch@infradead.org>,
	Chase Venters <chase.venters@clientec.com>
Subject: Re: [take19 1/4] kevent: Core files.
Date: Tue, 17 Oct 2006 17:42:07 +0400	[thread overview]
Message-ID: <20061017134206.GC20225@2ka.mipt.ru> (raw)
In-Reply-To: <200610171519.37051.dada1@cosmosbay.com>

On Tue, Oct 17, 2006 at 03:19:36PM +0200, Eric Dumazet (dada1@cosmosbay.com) wrote:
> On Tuesday 17 October 2006 12:39, Evgeniy Polyakov wrote:
> 
> > I can add such notification, but its existense _is_ the broken design.
> > After such condition happend, all new events will dissapear (although
> > they are still accessible through usual queue) from mapped buffer.
> >
> > While writing this I have come to the idea on how to imrove the case of
> > the size of mapped buffer - we can make it with limited size, and when
> > it is full, some bit will be set in the shared area and obviously no new
> > events can be added there, but when user commits some events from that
> > buffer (i.e. says to kernel that appropriate kevents can be freed or
> > requeued according to theirs flags), new ready events from ready queue
> > can be copied into mapped buffer.
> >
> > It still does not solve (and I do insist that it is broken behaviour)
> > the case when kernel is going to generate infinite number of events for
> > one requested by userspace (as in case of generating new 'data_has_arrived'
> > event when new byte has been received).
> 
> Behavior is not broken. It's quite usefull and works 99.9999% of time.
>
> I was trying to suggest you but you missed my point.
> 
> You dont want to use a bit, but a full sequence counter, 32bits.
> 
> A program may handle XXX.XXX handles, but use a 4096 entries ring 
> buffer 'only'.
> 
> The user program keeps a local copy of a special word 
> named 'ring_buffer_full_counter'
> 
> Each time the kernel cannot queue an event in the ring buffer, it increase 
> the "ring_buffer_was_full_counter" (exported to user app in the mmap view)
> 
> When the user application notice the kernel 
> changed "ring_buffer_was_full_counter" it does a full scan of all file 
> handles (preferably using poll() to get all relevant info in one syscall) :

I.e. to scan the rest of the xxx.xxx events?

> do {
>    if (read_event_from_mmap()) {handle_event(fd); continue;}
>    /* ring buffer is empty, check if we missed some events */
>    if (unlikely(mmap->ring_buffer_full_counter !=  
> my_ring_buffer_full_counter)) {
> 	my_ring_buffer_full_counter = mmap->ring_buffer_full_counter;
> 	/* slow PATH */
> 	/* can use a big poll() for example, or just a loop without poll() */
> 	for_all_file_desc_do() {
> 		check if some event/data is waiting on THIS fd
> 		}
> 	/* 
> 	}
>     else  syscall_wait_for_one_available_kevent(queue)
> }
> 
> This is how a program can recover. If ring buffer has a reasonable size, this 
> kind of event should not happen very frequently. If it does (because events 
> continue to fill ring_buffer during recovery and might hit FULL again), maybe 
> a smart program is able to resize the ring_buffer, and start using it after 
> yet another recovery pass.
> If not, we dont care, because a big poll() give us many ready file-descriptors 
> in one syscall, and maybe this is much better than kevent/epoll when XX.XXX 
> events are ready.

What about the case, which I described in other e-mail, when in case of
the full ring buffer, no new events are written there, and when
userspace commits (i.e. marks as ready to be freed or requeued by kernel) 
some events, new ones will be copied from ready queue into the buffer?

> Eric

-- 
	Evgeniy Polyakov

  reply	other threads:[~2006-10-17 13:42 UTC|newest]

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <115a6230591036@2ka.mipt.ru>
2006-09-12  8:41 ` [take18 0/4] kevent: Generic event handling mechanism Evgeniy Polyakov
2006-09-12  8:41   ` [take18 1/4] kevent: Core files Evgeniy Polyakov
2006-09-12  8:41     ` [take18 2/4] kevent: poll/select() notifications Evgeniy Polyakov
2006-09-12  8:41       ` [take18 3/4] kevent: Socket notifications Evgeniy Polyakov
2006-09-12  8:41         ` [take18 4/4] kevent: Timer notifications Evgeniy Polyakov
2006-09-20  9:35 ` [take19 0/4] kevent: Generic event handling mechanism Evgeniy Polyakov
2006-09-20  9:35   ` [take19 1/4] kevent: Core files Evgeniy Polyakov
2006-09-20  9:35     ` [take19 2/4] kevent: poll/select() notifications Evgeniy Polyakov
2006-09-20  9:35       ` [take19 3/4] kevent: Socket notifications Evgeniy Polyakov
2006-09-20  9:35         ` [take19 4/4] kevent: Timer notifications Evgeniy Polyakov
2006-10-04  6:34     ` [take19 1/4] kevent: Core files Ulrich Drepper
2006-10-04  6:48       ` Evgeniy Polyakov
2006-10-04 17:57         ` Ulrich Drepper
2006-10-05  8:57           ` Evgeniy Polyakov
2006-10-05  9:56             ` Eric Dumazet
2006-10-05 10:21               ` Evgeniy Polyakov
2006-10-05 10:45                 ` Eric Dumazet
2006-10-05 10:55                   ` Evgeniy Polyakov
2006-10-05 12:09                     ` Eric Dumazet
2006-10-05 12:37                       ` Evgeniy Polyakov
2006-10-15 23:22                         ` Ulrich Drepper
2006-10-16  7:33                           ` Evgeniy Polyakov
2006-10-16 10:16                             ` Ulrich Drepper
2006-10-16 11:23                               ` Evgeniy Polyakov
2006-10-17  5:10                           ` Johann Borck
2006-10-17  5:59                             ` Chase Venters
2006-10-17 10:42                               ` Evgeniy Polyakov
2006-10-17 13:12                                 ` Chase Venters
2006-10-17 13:35                                   ` Evgeniy Polyakov
2006-10-17 10:39                             ` Evgeniy Polyakov
2006-10-17 13:19                               ` Eric Dumazet
2006-10-17 13:42                                 ` Evgeniy Polyakov [this message]
2006-10-17 13:52                                   ` Eric Dumazet
2006-10-17 14:07                                     ` Evgeniy Polyakov
2006-10-17 14:25                                       ` Eric Dumazet
2006-10-17 15:09                                         ` Evgeniy Polyakov
2006-10-17 15:32                                           ` Eric Dumazet
2006-10-17 16:01                                             ` Evgeniy Polyakov
2006-10-17 16:26                                               ` Eric Dumazet
2006-10-17 16:35                                                 ` Evgeniy Polyakov
2006-10-17 16:45                                                   ` Eric Dumazet
2006-10-18  4:10                                                     ` Evgeniy Polyakov
2006-10-18  4:45                                                       ` Eric Dumazet
2006-10-17 15:33                                         ` Hans Henrik Happe
2006-10-05 14:01                 ` Hans Henrik Happe
2006-10-05 14:15                   ` Evgeniy Polyakov
2006-10-05 15:07                     ` Hans Henrik Happe
2006-09-22 19:22   ` [take19 0/4] kevent: Generic event handling mechanism Andrew Morton
2006-09-23  4:23     ` Evgeniy Polyakov
2006-10-04  6:09       ` Ulrich Drepper
2006-10-04  6:10         ` Ulrich Drepper
2006-10-04  6:27           ` Evgeniy Polyakov
2006-10-04  6:24         ` Evgeniy Polyakov
2006-09-26 15:54     ` Christoph Hellwig
2006-09-27  4:46       ` Evgeniy Polyakov
2006-09-27 15:09   ` Evgeniy Polyakov
2006-10-04  4:50     ` Ulrich Drepper
2006-10-04  4:55       ` Evgeniy Polyakov
2006-10-04  7:33         ` Ulrich Drepper
2006-10-04  7:48           ` Evgeniy Polyakov
2006-10-04 17:20             ` Ulrich Drepper
2006-10-05  9:02               ` Evgeniy Polyakov
2006-10-05 14:45                 ` Ulrich Drepper
2006-10-06  8:36                   ` Evgeniy Polyakov
2006-10-15 22:43                     ` Ulrich Drepper
2006-10-16  7:23                       ` Evgeniy Polyakov
2006-10-16  9:59                         ` Ulrich Drepper
2006-10-16 10:38                           ` Evgeniy Polyakov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20061017134206.GC20225@2ka.mipt.ru \
    --to=johnpol@2ka.mipt.ru \
    --cc=akpm@osdl.org \
    --cc=chase.venters@clientec.com \
    --cc=dada1@cosmosbay.com \
    --cc=davem@davemloft.net \
    --cc=drepper@gmail.com \
    --cc=drepper@redhat.com \
    --cc=hch@infradead.org \
    --cc=johann.borck@densedata.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=zach.brown@oracle.com \
    --subject='Re: [take19 1/4] kevent: Core files.' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).