Re: Runs with Linux (tm)

Re: Runs with Linux (tm)

[Pavel Machek]

Hi!

> >>If even some "Linux-friendly" hardware manufacturers barely cooperate
> >>with the Linux comminuty now what makes you think this would work?
> >>   
> >>
> >
> >Nothing in life is guaranteed.  But at the very least, I think it would 
> >be a good step towards improving the Linux end user experience.  Instead 
> >of the unclear mess we have now (Is it supported?  Check with your 
> >vendor!), we would be able to say "Look for the Linux Certified logo".  
> >Combine that with a standard format for source code driver disks, and 
> >it would be a good step in the right direction.
> >
> > 
> >
> The problem as I see it:
> 
> A hardware vendor hires someone to write a driver.  The driver is 
> completed and submitted and finally makes it into the kernel.  It's 
> fully GPL and everyone is happy.  The hardware gets a "Native Linux 
> Support" logo.  The card goes out of favor and no one is interested in 
> maintaining the driver, it is marked obsolete and finally removed from 
> the kernel. ...the logo still suggests the hardware will work.

And? There's no problem. You still have GPLed driver, that is
reasonably nice (it was in kernel at one point), so you just fix that.
Don't overcomplicate this.

								Pavel
-- 
Thanks, Sharp!

Re: Fuzzy hash stuff.. (was Re: 2.1.xxx makes Electric Fence 22x slower)

[Jamie Lokier]

"David S. Miller" <davem@dm.cobaltmicro.com> wrote:
>As promised here is my work in progress fuzzy hash VMA lookup stuff.

On Tue, Aug 25, 1998 at 10:47:26PM +1000, Keith Owens wrote:
> Lots of code with very few comments snipped.  Come on Davem, make it
> understandable for us mere mortals.  If it took two people to fix
> quirks and bugs and converge the algorithm, surely a few notes on how
> it works would not go astray.

Nope, I can't see how it works either.

BTW, a splay tree would also be as fast as what we have now in the
common case, without need for a special one entry cache.  The root of
the tree automatically acts as the cache.

-- Jamie

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.altern.org/andrebalsa/doc/lkml-faq.html

Re: GPS Leap Second Scheduled!

[Jamie Lokier]

On Wed, Sep 09, 1998 at 09:35:59AM -0700, David Lang wrote:
> I am probably missing something, but can't you just ignore the leap second
> until you discover that the time is 1 sec off and then use the normal NTP
> procedure to get back to the exact time

Until the NTP procedure discovers and corrects this (a few minutes, plus
correction time), anything that expects synchronised time between
machines can go wrong.

Admittedly synchronisation isn't perfect anyway.

-- Jamie

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[Michael Shields]

In article <19980910073422.A13283@tantalophile.demon.co.uk>,
Jamie Lokier <lkd@tantalophile.demon.co.uk> wrote:
> On Wed, Sep 09, 1998 at 09:35:59AM -0700, David Lang wrote:
> > I am probably missing something, but can't you just ignore the leap second
> > until you discover that the time is 1 sec off and then use the normal NTP
> > procedure to get back to the exact time
> 
> Until the NTP procedure discovers and corrects this (a few minutes, plus
> correction time), anything that expects synchronised time between
> machines can go wrong.

NTP has the capability to know in advance that a leap second is
scheduled and act upon that at the correct time.

Check your logs the next time a leap second happens; xntpd does it.
-- 
Shields, CrossLink.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: Access to I/O-mapped / Memory-mapped resources

[Jamie Lokier]

On Wed, Dec 09, 1998 at 03:19:10PM -0800, Linux Lists wrote:
> I have a question: is there any reference in regards to how / when to use
> the virt_to_phys, virt_to_bus, ioremap, etc. ... functions, other than 
> /usr/src/linux/Documentation/IO-mapping.txt ?!? I'd like to understand it
> better, but this text has not been enough (for me, of course).

Well, the whole address thing is a bit messy anyway; I don't expect a
perfect understanding of it is even possible...

But in case it helps, try the document below.

> If there is no other way, I'll try to re-read it 1000 times to see if my
> understanding increases 1000 times as well ... ;)

Do that :-)

	linux/Documentation/IO-mapping.txt
        ----------------------------------

...is a little out of date but basically right.  Give it a read.
Then read this addendum; it might clarify things a bit.

Types of address
================

*bus* is an address you pass to devices.  E.g., what you'd write to a
PCI bus-mastering DMA device for its target address.  To access a bus
address from kernel C code, known as memory-mapped I/O, you must use
ioremap() to convert it to an *ioremap* address.  From C code, these
should always be accessed through readl(), writel() etc. and not as
ordinary memory references.  See <asm/io.h>.

*phys* is a CPU address after MMU translation.  It only appears in page
tables and things related to page tables.  Even this is hidden to some
extent because pte_page(*pte) returns a *virt* address despite appearances.
See <asm/pgtable.h>.

*virt* is a kernel direct-mapped address.  These are addresses you can
read and write from C, that correspond to main memory.  E.g., on x86,
the *virt* address 0xc0001000 means the 4097th byte of main memory.  See
<asm/page.h>.

There are other kinds of address too:

*user* addresses (such as passed to read() and write()) are
none of the above, and should always by accessed through get_user(),
put_user() etc.  See <asm/uaccess.h>.

*static* addresses are the addresses of functions and variables that are
declared in source code.  Because kernel code and modules are allocated
in various ways, you can't assume much about these addresses, but you
can always read and write them from kernel C code.

*vmalloc* addresses (returned by vmalloc()).  These are kernel virtual
address, which you can read and write from kernel C code.  But you can't
pass them to any of the virt_to_XXX macros, because they're *not* *virt*
addresses!  See <linux/vmalloc.h>.

*fixmap* addresses (returned by fix_to_virt()) (which has a misleading
name).  These are like *vmalloc* addresses: you can't pass them to the
virt_to_XXX macros, so they're _not_ *virt* addresses.  It's not very
clear if you're supposed to use readl() and writel() to access these.
See <asm/fixmap.h>.

*ioremap* addresses are returned by ioremap(), which takes a *bus*
address.  These have some similarity to *vmalloc* addresses, but you can
only use readl(), writel() etc. to access the device memory referred to
here.  Unhelpfully, just reading and writing these directly does work on
some architectures, and most older device drivers still do this.

Memory map
==========

The actual memory map varies a lot between architectures.  But since
someone asked, I'll give a quick summary of one particular memory map.
This example is for an i386 architecture: a particular 64MB Pentium II
dual processor box with PCI and an ISA bridge.

Virtual map
...........

This is what C code sees in user mode:

0x00000000-0xbfffffff  User space virtual memory mappings.

This is what C code sees in kernel mode:

0x00000000-0xbfffffff User space virtual memory mappings (current->mm context).
0xc0000000-0xc3ffffff 64MB kernel view of all of main memory, uses 4MB pages.
0xc4000000-0xc47fffff 8MB unmapped hole.
0xc4800000-0xffffbfff Kernel virtual mappings for vmalloc() and ioremap().
0xffffc000-0xffffcfff Memory mapped local APIC registers.
0xffffd000-0xffffdfff Memory mapped IO-APIC registers.

The 64MB view is subdivided like this (it depends on the PC's details):

0xc0000000-0xc00003ff Zero page, reserved for BIOS.
0xc0000000-0xc009ffff Low memory (first 640k minus zero page).
0xc00a0000-0xc00fffff Low memory-mapped I/O (especially VGA adapter) and ROMs.
0xc0100000-0xc3ffffff High memory (remaining 63MB).

The 64MB view at 0xc0000000 (= PAGE_OFFSET) is directly addressable main
memory.  This is simply memory addresses with PAGE_OFFSET added.  This
contains the main kernel image, and memory allocated with kmalloc(),
get_free_page() and the slab allocator.  Cached disk pages, network
buffers etc. are all addressed in this space.

The 64MB view is the *virt* addresses described earlier.  "Virtual" here
simply refers to the PAGE_OFFSET translation, nothing more.

The vmalloc() mappings are a different way to see this memory, used only
when a large, contiguous address range needs to be allocated.  This is
used to hold loaded modules amongst other things.

The ioremap() mappings occupy the same address range as the vmalloc()
mappings, but are a view onto memory-mapped I/O space (MMIO).  Not all
devices are mapped with ioremap() -- those in the low memory-mapped area
aren't.  In theory you are supposed to use readl(), writel(),
memcpy_fromio() etc. to access memory-mapped I/O space, but many older
drivers fail to do this and work fine on the current x86 implementation.

Physical map
............

Physical addresses are the result of virtual address translation on
board the CPU.  C code (and assembly code) doesn't see these directly,
but they are used in page tables, which control the address translation.

There is some confusion in the kernel page table code about whether the
physical addresses passed around are *phys* addresses (also known as
*linear*), or *virt* address, which are a restricted subset of *phys*
with PAGE_OFFSET added.

When setting entries, *phys* tends to be used, but when reading entries
*virt* tends to be returned.  This sometimes loses information, so
breaking some device drivers.  Perhaps those drivers are broken by
design anyway.

Bus map
.......

This view is completely different to the virtual address view, and the
*virt* view which is a subset of virtual addresses.  Bus addresses can
overlap virtual addresses in an arbitrary way, 

The bus map is the view seen by peripheral devices, like video cards and
disk controllers.  Although different from the CPU's physical map (which
is a sort of private bus map for the CPU), the bus addresses tend to be
consistent between different devices in a single machine.

On a PC, the bus map is arranged by the system BIOS at boot time,
according to rules of Plug'n'Play and other rules.  For the PCI bus,
regions of prefetchable and non-prefetchable memory are mixed
arbitrarily: there's no particularly significant address where one kind
stops and another starts.  (Although your BIOS might make it appear so).
You don't have to worry about the differences, as long as your BIOS
configured everything properly.

`lspci -vb' will show the bus addresses of all PCI devices on a system.

Memory mapped ISA cards tend to have rather low addresses (in the first
megabyte), while PCI cards can be mapped to all sorts of addresses, high
and low depending on the BIOS.

Non-PC architectures have different rules.

On an i386 architecture, bus addresses and *phys* addresses are the
same.  This is convenient but it does tend to hide some problems.  On
other architectures, these two are often different.

Devices with *bus* addresses are supposed to be memory mapped using
ioremap(), and then accessed using readl(), writel() etc.  Because none
of this was necessary on the i386 with the 2.0.x kernels, and the other
platforms weren't very well supported then, many older device drivers
simply access device bus addresses as if they were memory.

This poses big problems with some non-i386 architectures, which require
readl() etc. for the drivers to work.  These days it also poses problems
with the i386, because in 2.1.x kernels the memory layout was changed to
make communication between user space and kernel space more efficient.
As a result, ioremap() is required to get a virtual address which you
can pass to readl(), writel() etc.

Note: there is an ironic twist.  The virtual address returned by
ioremap() is not a *virt* address, so you can't expect meaningful
results if you pass it to virt_to_bus() or virt_to_phys().

Another note: at least on a PC, you don't need ioremap() to access
devices in the first 1MB of the bus address range.  This includes most
ISA devices (but not video cards).

I/O port map
............

This is similar to the bus map, but refers to I/O ports that are
accessed by special I/O instructions from the CPU, if it is an i386
based architecture.  For some other architectures, the I/O ports are
actually quite similar to memory-mapped I/O but using different
addresses.  Although some buses support more than 64k I/O ports, the
i386 architecture does not so this address range is restricted to
0x0000-0xffff.

`cat /proc/ioports' shows all the I/O ports used by drivers currently
loaded on a system.  `lspci -v' shows all the I/O ports used by PCI
devices.

Use inb(), outb() etc. to access I/O ports.  This has been required ever
since the earliest versions of Linux, so all drivers that use I/O ports
get this right.  There is no equivalent to ioremap().

Translating virtual addresses
=============================

Some people try to look up page tables to convert a *user* address or
*vmalloc* address to a *bus* address or *virt* address.  This works for
some things, but breaks others.  It makes a number of assumptions that
are incorrect and won't work when you want to use the driver in a new
way one day, or on a new architecture.

This mess will be cleaned up sometime in version 2.3.  So if you want it
cleaned up, perhaps the best way is to help ensure 2.2 is ready for
release.  Hint :-)

Hope this helps,
-- Jamie

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

Re: PROPOSAL: dot-proc interface [was: /proc stuff]

[Pavel Machek]

Hi!

> > > > It eats CPU, it's error-prone, and all in all it's just "wrong".
> > >
> > > How much of your CPU time is spent parsing /proc files?
> >
> > 30% of 486 if you run top... That's way too much and top is unusable
> > on slower machines.
> > "Not fast enough for showing processes" sounds wery wrong.
> 
> Is this time actually spent parsing ascii, or is it procfs
> walking all the page tables of all processes ? ;)

About 1:1, probably. Readdir of /proc and open/read/parse/close is 
pretty expensive.
								Pavel
-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

Re: RLIM_INFINITY inconsistency between archs

[Adam Sampson]

On Thu, Jul 27, 2000 at 12:39:51AM -0700, Linus Torvalds wrote:
> Is there some documentation file that I've not updated and that people
> are slavishly following outdated information in? I don't read the
> documentation myself, so I'd never notice ;)

Yes; the glibc installation instructions.

-- 

Adam Sampson
azz@gnu.org

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

Re: RLIM_INFINITY inconsistency between archs

[Adam Sampson]

On Thu, Jul 27, 2000 at 07:03:57PM +0200, Jamie Lokier wrote:
> But instead, how about a script: /lib/modules/VERSION/compile-module.
> The script would know where to find the kernel headers.  That could be
> /lib/modules/include for distributions, and /my/kernel/tree/include for
> folks who used `make modules_install' recently.

I'll second that suggestion. This kind of thing works very well indeed for
projects like Apache.

-- 

Adam Sampson
azz@gnu.org

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

Re: RLIM_INFINITY inconsistency between archs

[Miquel van Smoorenburg]

In article <cistron.20000728232030.C8868@gnu.org>,
Adam Sampson  <azz@gnu.org> wrote:
>On Thu, Jul 27, 2000 at 07:03:57PM +0200, Jamie Lokier wrote:
>> But instead, how about a script: /lib/modules/VERSION/compile-module.
>> The script would know where to find the kernel headers.  That could be
>> /lib/modules/include for distributions, and /my/kernel/tree/include for
>> folks who used `make modules_install' recently.
>
>I'll second that suggestion. This kind of thing works very well indeed for
>projects like Apache.

It is indeed a very good idea. The script could just spit out the
CFLAGS used for kernel compilation like this:

#! /bin/sh
cat <<EOF
-D__KERNEL__ -I/usr/src/linux-2.2.15/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DUTS_MACHINE='"i386"'
EOF

Then a module Makefile would be as simple as

# Set KVER manually if you want to compile against another kernel version
KVER=$(shell uname -r)
CFLAGS=$(shell /lib/modules/$(KVER)/kernel-config)

module.o: module.c module.h

I've tried this, it works.

Mike.
-- 
Cistron Certified Internetwork Expert #1. Think free speech; drink free beer.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

Re: Quota mods needed for journaled quota

[Nathan Scott]

hi Stephen,

On Oct 25,  6:42pm, Stephen C. Tweedie wrote:
> Subject: Quota mods needed for journaled quota
> ...
> All of these could be fixed very easily (at least for ext3) if it were
> possible for ext3 to install its own version of the superblock->dq_ops
> quota operations (which would just be simple wrappers around the
> existing quota calls).  However, the current sys_quotactl installs the
> default quota_ops into the superblock on quota_on without any chance
> for the filesystem to override it.
> 
> The addition of an "init_quota" method to the super_operations struct,
> with quota_on calling this and defaulting to installing the default
> quota_ops if the method is NULL, ought to be sufficient to let ext3
> get quotas right in all cases as far as I can see.
> 
> Comments?
> 

It might also/alternatively be generally useful to allow a
filesystem-specific implementation of quotactl itself - through
an additional member in the dquot_operations set of functions?

This would allow ext3 to do that which it needs to do differently
at Q_QUOTAON and would also allow Jan's changes to work in such
a way that both the current form of dquot structure and his new
version of dquots could be used together (his patches currently
change the ondisk dquot definition, which means the existing user
tools get back different structures to what they were expecting,
after issuing certain quotactl commands).

XFS also has its own, different ondisk format for dquots, which
it would like to pass over the quotactl interface - I imagine
filesystems coming from other OSs would too.  The quotactl
syscall is sufficiently generic - its alot like ioctl ;-) -
to allow any size/form of dquot to be passed back to userspace,
so a few new quotactl commands for Jan's new dquot structure
would allow the existing tools to continue to work & new user
tools could take advantage of his extensions (same again for XFS).

Anyway, hope this is useful input.


cheers.

ps: hmm - just realized something... is jank@redhat == jack@suse?

-- 
Nathan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Quota mods needed for journaled quota

[Nathan Scott]

hi Stephen,

On Oct 26, 11:00am, Stephen C. Tweedie wrote:
> Subject: Re: Quota mods needed for journaled quota
> ...
> > This would allow ext3 to do that which it needs to do differently
> > at Q_QUOTAON and would also allow Jan's changes to work in such
> > a way that both the current form of dquot structure and his new
> > version of dquots could be used together
> 
> Adding the init_quota hook would do that, as the filesystem will be
> able to install its own dq_ops methods during the init so we get the
> flexibility you are asking for anyway.
> 

Hmmm ... I'm not so sure.  In order to have the flexibility
of filesystem-specific dquot formats, the struct dquot would
need to become more like struct inode/super_block, i.e. not
hardcoding the ondisk structure into the incore structure
(using a union and a generic pointer, as inode/super_block do).

The DQUOT_SYNC mechanism would need to be able to be overridden
per-filesystem also.  It isn't really as cut-and-dried as "per-
filesystem" either, because an ext2/3 filesystem might make use
of either the original dquot format or Jan's newer format, either
at mount time or even after doing a quota_off & quota_on with a
new quota file format (that would be quite clean).

But I've sidetracked completely from what you were originally
talking about now, which had nothing to do with a different
ondisk format at all.

cheers.

-- 
Nathan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [patch] kernel/module.c (plus gratuitous rant)

[Pavel Machek]

Hi!

> > if the person who sent you the -pre4 patch against module.c
> > had Cc:'ed this mailing list then your kernel would do
> > something useful when compiled with gcc-2.7.2.3.
> 
> It seems that gcc-2.7.2.3 is terminally ill. I'd rather change
> Documentation/Changes, and just document the fact.
> 
> These kinds of subtle work-arounds for gcc bugs are not really acceptable,
> nor is it worthwhile complaining when somebody does development with a gcc
> that is _not_ broken, and doesn't notice that some random gcc bug breaks
> the kernel for others.

Would it be possible to keep 2.7.2.3? You still need 2.7.2.3 to
reliably compile 2.0.X (and maybe even 2.2.all-but-latest?).
								Pavel
-- 
I'm pavel@ucw.cz. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at discuss@linmodems.org
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [patch] kernel/module.c (plus gratuitous rant)

[Jeff Garzik]

Pavel Machek wrote:
> Would it be possible to keep 2.7.2.3? You still need 2.7.2.3 to
> reliably compile 2.0.X (and maybe even 2.2.all-but-latest?).

What fails, when you use egcs-1.1.2 to build 2.0.x or early 2.2.x?

Maybe they need -fno-strict-aliasing... is that what you are referring
to?

Regards,

	Jeff



-- 
Jeff Garzik                    | "Mind if I drive?"  -Sam
Building 1024                  | "Not if you don't mind me clawing at
the
MandrakeSoft                   |  dash and screaming like a
cheerleader."
                               |      -Max
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [patch] kernel/module.c (plus gratuitous rant)

[Alan Cox]

> Would it be possible to keep 2.7.2.3? You still need 2.7.2.3 to
> reliably compile 2.0.X (and maybe even 2.2.all-but-latest?).

There has only been one know egcs 1.1 build problem found in the last 9 
months or so (the fpu emu one). I really dont think using egcs 1.1.2 to build
2.2 kernels is a problem. In fact its probably the default nowdays

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [patch] kernel/module.c (plus gratuitous rant)

[Alan Cox]

> Pavel Machek wrote:
> > Would it be possible to keep 2.7.2.3? You still need 2.7.2.3 to
> > reliably compile 2.0.X (and maybe even 2.2.all-but-latest?).
> 
> What fails, when you use egcs-1.1.2 to build 2.0.x or early 2.2.x?

egcs miscompiles inlined strstr. It gets combined with bad asm constraints
to mean that 2.0 and earlier 2.2 will crash when fed the right (wrong ?) 
sequence of FPU ops to software emulate


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [patch] kernel/module.c (plus gratuitous rant)

[Keith Owens]

On Fri, 27 Oct 2000 19:45:13 +0200, 
Pavel Machek <pavel@suse.cz> wrote:
>Would it be possible to keep 2.7.2.3? You still need 2.7.2.3 to
>reliably compile 2.0.X (and maybe even 2.2.all-but-latest?).

You can have multiple versions of gcc installed, just select the one to
use when you compile the kernel.

CC=gcc-2723 make 2.0 kernel
CC=gcc-2723 make 2.2 kernel
CC=egcs make 2.4 kernel

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [patch] kernel/module.c (plus gratuitous rant)

[Dominik Kubla]

On Sat, Oct 28, 2000 at 12:00:43PM +1100, Keith Owens wrote:
> On Fri, 27 Oct 2000 19:45:13 +0200, 
> Pavel Machek <pavel@suse.cz> wrote:
> >Would it be possible to keep 2.7.2.3? You still need 2.7.2.3 to
> >reliably compile 2.0.X (and maybe even 2.2.all-but-latest?).
> 
> You can have multiple versions of gcc installed, just select the one to
> use when you compile the kernel.
> 
> CC=gcc-2723 make 2.0 kernel
> CC=gcc-2723 make 2.2 kernel
> CC=egcs make 2.4 kernel

Even simpler: "gcc -V 2.7.2.3" or "gcc -V 2.95.2" or whatever...

Yours,
  Dominik Kubla
-- 
http://petition.eurolinux.org/index_html - No Software Patents In Europe!
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [patch] kernel/module.c (plus gratuitous rant)

[Richard Henderson]

On Sat, Oct 28, 2000 at 01:15:58PM +0200, Dominik Kubla wrote:
> Even simpler: "gcc -V 2.7.2.3" or "gcc -V 2.95.2" or whatever...

Which was a nice idea, but it doesn't actually work.  Changes
in spec file format between versions makes this fall over.


r~
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [patch] kernel/module.c (plus gratuitous rant)

[Dominik Kubla]

On Sat, Oct 28, 2000 at 05:27:00PM -0700, Richard Henderson wrote:
> On Sat, Oct 28, 2000 at 01:15:58PM +0200, Dominik Kubla wrote:
> > Even simpler: "gcc -V 2.7.2.3" or "gcc -V 2.95.2" or whatever...
> 
> Which was a nice idea, but it doesn't actually work.  Changes
> in spec file format between versions makes this fall over.

Wow. So much for reading the manual... well, that's considered 
cheating anyway, isn't it?

Dominik
-- 
http://petition.eurolinux.org/index_html - No Software Patents In Europe!
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Alan,

I don't know what all changes you guys did to 2.2.18pre, but the LAN I/O
performance absolutely rocks on our tests here vs. NetWare.  Good job. 
It's still got some problems with NFS (I am seeing a few RPC timeout
errors) so I am backreving to 2.2.17 for the Ute-NWFS release next week,
but it's most impressive.

:-)

Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [patch] kernel/module.c (plus gratuitous rant)

[Rusty Russell]

In message <4309.972694843@ocs3.ocs-net> you write:
> On Fri, 27 Oct 2000 19:45:13 +0200, 
> Pavel Machek <pavel@suse.cz> wrote:
> >Would it be possible to keep 2.7.2.3? You still need 2.7.2.3 to
> >reliably compile 2.0.X (and maybe even 2.2.all-but-latest?).
> 
> You can have multiple versions of gcc installed, just select the one to
> use when you compile the kernel.
> 
> CC=gcc-2723 make 2.0 kernel
> CC=gcc-2723 make 2.2 kernel
> CC=egcs make 2.4 kernel

No, environment doesn't override make variables by default.  This
works on any shell:

	make CC=egcs <targets>

Rusty.
--
Hacking time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 12:04:23AM +0000, Alan Cox wrote:
> > It's still got some problems with NFS (I am seeing a few RPC timeout
> > errors) so I am backreving to 2.2.17 for the Ute-NWFS release next week,
> > but it's most impressive.
> 
> Can you send a summary of the NFS reports to nfs-devel@linux.kernel.org

Yes.  I just went home, so I am emailing from my house.  I'll post late 
tonight or in the morning.  Performance on 100Mbit with NFS going 
Linux->Linux is getting better throughput than IPX NetWare Client -> 
NetWare 5.x on the same network by @ 3%.  When you start loading up a 
Linux server, it drops off sharply and NetWare keeps scaling, however, 
this does indicate that the LAN code paths are equivalent relative to 
latency vs. MSM/TSM/HSM in NetWare.  NetWare does better caching 
(but we'll fix this in Linux next).  I think the ring transitions to 
user space daemons are what are causing the scaling problems 
Linux vs. NetWare.

:-)

Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Andi Kleen]

On Sun, Oct 29, 2000 at 06:35:31PM -0700, Jeff V. Merkey wrote:
> On Mon, Oct 30, 2000 at 12:04:23AM +0000, Alan Cox wrote:
> > > It's still got some problems with NFS (I am seeing a few RPC timeout
> > > errors) so I am backreving to 2.2.17 for the Ute-NWFS release next week,
> > > but it's most impressive.
> > 
> > Can you send a summary of the NFS reports to nfs-devel@linux.kernel.org
> 
> Yes.  I just went home, so I am emailing from my house.  I'll post late 
> tonight or in the morning.  Performance on 100Mbit with NFS going 
> Linux->Linux is getting better throughput than IPX NetWare Client -> 
> NetWare 5.x on the same network by @ 3%.  When you start loading up a 
> Linux server, it drops off sharply and NetWare keeps scaling, however, 
> this does indicate that the LAN code paths are equivalent relative to 
> latency vs. MSM/TSM/HSM in NetWare.  NetWare does better caching 
> (but we'll fix this in Linux next).  I think the ring transitions to 
> user space daemons are what are causing the scaling problems 
> Linux vs. NetWare.

There are no user space daemons involved in the knfsd fast path, only in slow paths
like mounting.
The main problem I think in knfsd are the numerous copies of the data (e.g. 2+checksumming for
RX with fragments, upto 4 in some specific configurations). They're unfortunately 
not trivial to fix. TX is a bit better, it does only one copy usually out of
the page cache. For RX it also helps to have a network card that supports hardware
checksumming.



-Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 07:47:00AM +0100, Andi Kleen wrote:
> On Sun, Oct 29, 2000 at 06:35:31PM -0700, Jeff V. Merkey wrote:
> > On Mon, Oct 30, 2000 at 12:04:23AM +0000, Alan Cox wrote:
> > > > It's still got some problems with NFS (I am seeing a few RPC timeout
> > > > errors) so I am backreving to 2.2.17 for the Ute-NWFS release next week,
> > > > but it's most impressive.
> > > 
> > > Can you send a summary of the NFS reports to nfs-devel@linux.kernel.org
> > 
> > Yes.  I just went home, so I am emailing from my house.  I'll post late 
> > tonight or in the morning.  Performance on 100Mbit with NFS going 
> > Linux->Linux is getting better throughput than IPX NetWare Client -> 
> > NetWare 5.x on the same network by @ 3%.  When you start loading up a 
> > Linux server, it drops off sharply and NetWare keeps scaling, however, 
> > this does indicate that the LAN code paths are equivalent relative to 
> > latency vs. MSM/TSM/HSM in NetWare.  NetWare does better caching 
> > (but we'll fix this in Linux next).  I think the ring transitions to 
> > user space daemons are what are causing the scaling problems 
> > Linux vs. NetWare.
> 
> There are no user space daemons involved in the knfsd fast path, only in slow paths
> like mounting.

So why is it spawning off nfsd servers in user space?  I did notice that all
the connect logic is down in the kernel with the RPC stuff in xprt.c, and this
looks to be pretty fast.  I know about the checksumming problem with TCPIP.
But cycles are cheap on todays processors, so even with this overhead, it 
could still get faster.  IPX uses small packets that are less wire efficient 
since the ratio of header size to payload is larger than what NFS in Linux
is doing, plus there's more of them for an equivalent data transfer, even
with packet burst.   IPX would tend to be faster if there were multiple 
routers involved, since the latency of smaller packets would be less and
IPX never has to deal with the problem of fragmentation.

Is there any CR3 reloading or tasking switching going on for each interrupt
that comes in you know of, BTW?  EMON stats show the server's bus utilization
to get disproportiantely higher in Linux than NetWare 5.x and when it hits
60% of total clock cycles, Linux starts dropping off.  NetWare 5.x is 1/8 
the bus utilitization, which would suggest clocks are being used for something
other than pushing packets in and out of the box (the checksumming is done 
in the tcp code when it copies the fragments, which is very low overhead).

Jeff  


> The main problem I think in knfsd are the numerous copies of the data (e.g. 2+checksumming for
> RX with fragments, upto 4 in some specific configurations). They're unfortunately 
> not trivial to fix. TX is a bit better, it does only one copy usually out of
> the page cache. For RX it also helps to have a network card that supports hardware
> checksumming.
> 
> 
> 
> -Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Andi Kleen]

On Sun, Oct 29, 2000 at 11:58:21PM -0700, Jeff V. Merkey wrote:
> On Mon, Oct 30, 2000 at 07:47:00AM +0100, Andi Kleen wrote:
> > On Sun, Oct 29, 2000 at 06:35:31PM -0700, Jeff V. Merkey wrote:
> > > On Mon, Oct 30, 2000 at 12:04:23AM +0000, Alan Cox wrote:
> > > > > It's still got some problems with NFS (I am seeing a few RPC timeout
> > > > > errors) so I am backreving to 2.2.17 for the Ute-NWFS release next week,
> > > > > but it's most impressive.
> > > > 
> > > > Can you send a summary of the NFS reports to nfs-devel@linux.kernel.org
> > > 
> > > Yes.  I just went home, so I am emailing from my house.  I'll post late 
> > > tonight or in the morning.  Performance on 100Mbit with NFS going 
> > > Linux->Linux is getting better throughput than IPX NetWare Client -> 
> > > NetWare 5.x on the same network by @ 3%.  When you start loading up a 
> > > Linux server, it drops off sharply and NetWare keeps scaling, however, 
> > > this does indicate that the LAN code paths are equivalent relative to 
> > > latency vs. MSM/TSM/HSM in NetWare.  NetWare does better caching 
> > > (but we'll fix this in Linux next).  I think the ring transitions to 
> > > user space daemons are what are causing the scaling problems 
> > > Linux vs. NetWare.
> > 
> > There are no user space daemons involved in the knfsd fast path, only in slow paths
> > like mounting.
> 
> So why is it spawning off nfsd servers in user space?  I did notice that all

They just provide a process context, the actual work is only done in kernel mode.

> the connect logic is down in the kernel with the RPC stuff in xprt.c, and this
> looks to be pretty fast.  I know about the checksumming problem with TCPIP.
> But cycles are cheap on todays processors, so even with this overhead, it 
> could still get faster.  IPX uses small packets that are less wire efficient 
> since the ratio of header size to payload is larger than what NFS in Linux
> is doing, plus there's more of them for an equivalent data transfer, even
> with packet burst.   IPX would tend to be faster if there were multiple 
> routers involved, since the latency of smaller packets would be less and
> IPX never has to deal with the problem of fragmentation.
> 
> Is there any CR3 reloading or tasking switching going on for each interrupt
> that comes in you know of, BTW?  EMON stats show the server's bus utilization

No, interrupts do not change CR3.

One problem in Linux 2.2 is that kernel threads reload their VM on context switch
(that would include the nfsd thread), this should be fixed in 2.4 with lazy mm.
Hmm actually it should be only fixed for true kernel threads that have been started 
with kernel_thread(), the "pseudo kernel threads" like nfsd uses probably do not 
get that optimization because they don't set their MM to init_mm. 

> to get disproportiantely higher in Linux than NetWare 5.x and when it hits
> 60% of total clock cycles, Linux starts dropping off.  NetWare 5.x is 1/8 

I think that can be explained by the copying.

To be sure you could e.g. use the ktrace patch from IKD, it will give you
cycle accurate traces of execution of functions.

> the bus utilitization, which would suggest clocks are being used for something
> other than pushing packets in and out of the box (the checksumming is done 
> in the tcp code when it copies the fragments, which is very low overhead).

No, unfortunately not. NFS uses UDP and the UDP defragmenting doesn't do copy-checksum
currently.


-Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 08:08:58AM +0100, Andi Kleen wrote:
> On Sun, Oct 29, 2000 at 11:58:21PM -0700, Jeff V. Merkey wrote:
> > On Mon, Oct 30, 2000 at 07:47:00AM +0100, Andi Kleen wrote:
> > > On Sun, Oct 29, 2000 at 06:35:31PM -0700, Jeff V. Merkey wrote:
> > > > On Mon, Oct 30, 2000 at 12:04:23AM +0000, Alan Cox wrote:
> > > > > > It's still got some problems with NFS (I am seeing a few RPC timeout
> > > > > > errors) so I am backreving to 2.2.17 for the Ute-NWFS release next week,
> > > > > > but it's most impressive.
> > > > > 
> > > > > Can you send a summary of the NFS reports to nfs-devel@linux.kernel.org
> > > > 
> > > > Yes.  I just went home, so I am emailing from my house.  I'll post late 
> > > > tonight or in the morning.  Performance on 100Mbit with NFS going 
> > > > Linux->Linux is getting better throughput than IPX NetWare Client -> 
> > > > NetWare 5.x on the same network by @ 3%.  When you start loading up a 
> > > > Linux server, it drops off sharply and NetWare keeps scaling, however, 
> > > > this does indicate that the LAN code paths are equivalent relative to 
> > > > latency vs. MSM/TSM/HSM in NetWare.  NetWare does better caching 
> > > > (but we'll fix this in Linux next).  I think the ring transitions to 
> > > > user space daemons are what are causing the scaling problems 
> > > > Linux vs. NetWare.
> > > 
> > > There are no user space daemons involved in the knfsd fast path, only in slow paths
> > > like mounting.
> > 
> > So why is it spawning off nfsd servers in user space?  I did notice that all
> 
> They just provide a process context, the actual work is only done in kernel mode.
> 
> > the connect logic is down in the kernel with the RPC stuff in xprt.c, and this
> > looks to be pretty fast.  I know about the checksumming problem with TCPIP.
> > But cycles are cheap on todays processors, so even with this overhead, it 
> > could still get faster.  IPX uses small packets that are less wire efficient 
> > since the ratio of header size to payload is larger than what NFS in Linux
> > is doing, plus there's more of them for an equivalent data transfer, even
> > with packet burst.   IPX would tend to be faster if there were multiple 
> > routers involved, since the latency of smaller packets would be less and
> > IPX never has to deal with the problem of fragmentation.
> > 
> > Is there any CR3 reloading or tasking switching going on for each interrupt
> > that comes in you know of, BTW?  EMON stats show the server's bus utilization
> 
> No, interrupts do not change CR3.
> 
> One problem in Linux 2.2 is that kernel threads reload their VM on context switch
> (that would include the nfsd thread), this should be fixed in 2.4 with lazy mm.

When you say it reloads it's VM, you mean it reloads the CR3 register?  
This will cost you about 15 clocks up front, and about 150 clocks over
time as each TLB is reloaded (plus is will do LOCK# assertions invisibly
underneath for PTE fetches) One major difference is that in NetWare, 
CR3 does not ever get changed in any network I/O fast paths, and none of 
the TCP or IPX processes exist in user space, so there's never this 
background overhead with page tables being loaded in and out.  CR3 only 
gets mucked with when someone allocates memory and pages in an address 
frame (when memory is alloced and freed).

The user space activity is what's causing this, plus the copying.  Is there
an easy way to completely disable multiple PTE/PDE address spaces and just
map the entire address space linear (like NetWare) with a compile option 
so I could do a true apples to apples comparison?

Jeff


> Hmm actually it should be only fixed for true kernel threads that have been started 
> with kernel_thread(), the "pseudo kernel threads" like nfsd uses probably do not 
> get that optimization because they don't set their MM to init_mm. 
> 
> > to get disproportiantely higher in Linux than NetWare 5.x and when it hits
> > 60% of total clock cycles, Linux starts dropping off.  NetWare 5.x is 1/8 
> 
> I think that can be explained by the copying.

It's more.  I am seeing tons of segment register reloads, which are very 
heavy, and atomic reads of PTE entries.

> 
> To be sure you could e.g. use the ktrace patch from IKD, it will give you
> cycle accurate traces of execution of functions

I'll look at this.

> > the bus utilitization, which would suggest clocks are being used for something
> > other than pushing packets in and out of the box (the checksumming is done 
> > in the tcp code when it copies the fragments, which is very low overhead).
> 
> No, unfortunately not. NFS uses UDP and the UDP defragmenting doesn't do copy-checksum

Correct.  You're right -- it's in the tcp code, however, I remember going over
this when I was reviewing Alan's code -- that's what I was think of.

Jeff



> currently.
> 
> 
> -Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 09:26:59AM +0100, Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Andi Kleen wrote:
> 
> > One problem in Linux 2.2 is that kernel threads reload their VM on
> > context switch (that would include the nfsd thread), this should be
> > fixed in 2.4 with lazy mm. Hmm actually it should be only fixed for
> > true kernel threads that have been started with kernel_thread(), the
> > "pseudo kernel threads" like nfsd uses probably do not get that
> > optimization because they don't set their MM to init_mm.
> 
> yes, but for this there is an explicit mechanizm to lazy-MM during lengthy
> system calls, an example is in buffer.c:
> 
>                 user_mm = start_lazy_tlb();
>                 error = sync_old_buffers();
>                 end_lazy_tlb(user_mm);
> 
> > > to get disproportiantely higher in Linux than NetWare 5.x and when it hits
> > > 60% of total clock cycles, Linux starts dropping off.  NetWare 5.x is 1/8 
> > 
> > I think that can be explained by the copying.
> 
> yes. Constant copying contaminates the L1/L2 caches and creates dirty
> cachelines all around the place. Fixed in 2.4 + TUX ;-)
> 

Ingo, we need a build option to completely disable multiple address spaces
for a start, and just map everything to a linear address space.  This 
will eliminate the overhead of the CR3 activity.  The use of segment registers
for copy_to_user, etc. causes segment register reloads, which are very 
heavyweight on Intel.  

Is there an option to map Linux into a flat address space like NetWare so
I can do an apples to apples comparison of raw LAN I/O scaling?   


> 	Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

/ on ramfs, possible?

[Anders Eriksson]

[-- Attachment #1: Type: text/plain, Size: 316 bytes --]


I want my / to be a ramfs filesystem. I intend to populate it from an 
initrd image, and then remount / as the ramfs filesystem. Is that at 
all possible? The way I see it the kernel requires / on a device 
(major,minor) or nfs.

Am I out of luck using ramfs as /? If it's easy to fix, how do I fix it?

/Anders




[-- Attachment #2: Type: application/pgp-signature, Size: 235 bytes --]

Re: / on ramfs, possible?

[H. Peter Anvin]

Followup to:  <200010300727.IAA12250@hell.wii.ericsson.net>
By author:    Anders Eriksson <aer-list@mailandnews.com>
In newsgroup: linux.dev.kernel
> 
> I want my / to be a ramfs filesystem. I intend to populate it from an 
> initrd image, and then remount / as the ramfs filesystem. Is that at 
> all possible? The way I see it the kernel requires / on a device 
> (major,minor) or nfs.
> 
> Am I out of luck using ramfs as /? If it's easy to fix, how do I fix it?
> 

Use pivot_root instead of the initrd stuff in /proc/sys.

	-hpa

-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Andi Kleen]

On Mon, Oct 30, 2000 at 12:16:46AM -0700, Jeff V. Merkey wrote:
> On Mon, Oct 30, 2000 at 08:08:58AM +0100, Andi Kleen wrote:
> > On Sun, Oct 29, 2000 at 11:58:21PM -0700, Jeff V. Merkey wrote:
> > > On Mon, Oct 30, 2000 at 07:47:00AM +0100, Andi Kleen wrote:
> > > > On Sun, Oct 29, 2000 at 06:35:31PM -0700, Jeff V. Merkey wrote:
> > > > > On Mon, Oct 30, 2000 at 12:04:23AM +0000, Alan Cox wrote:
> > > > > > > It's still got some problems with NFS (I am seeing a few RPC timeout
> > > > > > > errors) so I am backreving to 2.2.17 for the Ute-NWFS release next week,
> > > > > > > but it's most impressive.
> > > > > > 
> > > > > > Can you send a summary of the NFS reports to nfs-devel@linux.kernel.org
> > > > > 
> > > > > Yes.  I just went home, so I am emailing from my house.  I'll post late 
> > > > > tonight or in the morning.  Performance on 100Mbit with NFS going 
> > > > > Linux->Linux is getting better throughput than IPX NetWare Client -> 
> > > > > NetWare 5.x on the same network by @ 3%.  When you start loading up a 
> > > > > Linux server, it drops off sharply and NetWare keeps scaling, however, 
> > > > > this does indicate that the LAN code paths are equivalent relative to 
> > > > > latency vs. MSM/TSM/HSM in NetWare.  NetWare does better caching 
> > > > > (but we'll fix this in Linux next).  I think the ring transitions to 
> > > > > user space daemons are what are causing the scaling problems 
> > > > > Linux vs. NetWare.
> > > > 
> > > > There are no user space daemons involved in the knfsd fast path, only in slow paths
> > > > like mounting.
> > > 
> > > So why is it spawning off nfsd servers in user space?  I did notice that all
> > 
> > They just provide a process context, the actual work is only done in kernel mode.
> > 
> > > the connect logic is down in the kernel with the RPC stuff in xprt.c, and this
> > > looks to be pretty fast.  I know about the checksumming problem with TCPIP.
> > > But cycles are cheap on todays processors, so even with this overhead, it 
> > > could still get faster.  IPX uses small packets that are less wire efficient 
> > > since the ratio of header size to payload is larger than what NFS in Linux
> > > is doing, plus there's more of them for an equivalent data transfer, even
> > > with packet burst.   IPX would tend to be faster if there were multiple 
> > > routers involved, since the latency of smaller packets would be less and
> > > IPX never has to deal with the problem of fragmentation.
> > > 
> > > Is there any CR3 reloading or tasking switching going on for each interrupt
> > > that comes in you know of, BTW?  EMON stats show the server's bus utilization
> > 
> > No, interrupts do not change CR3.
> > 
> > One problem in Linux 2.2 is that kernel threads reload their VM on context switch
> > (that would include the nfsd thread), this should be fixed in 2.4 with lazy mm.
> 
> When you say it reloads it's VM, you mean it reloads the CR3 register?  

Yes.

> This will cost you about 15 clocks up front, and about 150 clocks over
> time as each TLB is reloaded (plus is will do LOCK# assertions invisibly
> underneath for PTE fetches) One major difference is that in NetWare, 
> CR3 does not ever get changed in any network I/O fast paths, and none of 
> the TCP or IPX processes exist in user space, so there's never this 
> background overhead with page tables being loaded in and out.  CR3 only 
> gets mucked with when someone allocates memory and pages in an address 
> frame (when memory is alloced and freed).
> 
> The user space activity is what's causing this, plus the copying.  Is there
> an easy way to completely disable multiple PTE/PDE address spaces and just
> map the entire address space linear (like NetWare) with a compile option 
> so I could do a true apples to apples comparison?

No. In 2.4 you could probably use the on demand lazy vm mechanism ingo described for
the nfsd processes. In 2.2 it is a bit more tricky, if I remember right lazy mm needed
quite a few changes.

But before doing too many changes i would first verify if that is really the problem.


> > Hmm actually it should be only fixed for true kernel threads that have been started 
> > with kernel_thread(), the "pseudo kernel threads" like nfsd uses probably do not 
> > get that optimization because they don't set their MM to init_mm. 
> > 
> > > to get disproportiantely higher in Linux than NetWare 5.x and when it hits
> > > 60% of total clock cycles, Linux starts dropping off.  NetWare 5.x is 1/8 
> > 
> > I think that can be explained by the copying.
> 
> It's more.  I am seeing tons of segment register reloads, which are very 
> heavy, and atomic reads of PTE entries.

PTEs are read for aging on memory pressure in vmscan.

segment register reload happen on interrupt entry, to load the kernel cs/ds (are they that
costly?). If it was really costly you could probably check in interrupt entry if you're
already running in kernel space and skip it.

> 
> > > the bus utilitization, which would suggest clocks are being used for something
> > > other than pushing packets in and out of the box (the checksumming is done 
> > > in the tcp code when it copies the fragments, which is very low overhead).
> > 
> > No, unfortunately not. NFS uses UDP and the UDP defragmenting doesn't do copy-checksum
> 
> Correct.  You're right -- it's in the tcp code, however, I remember going over
> this when I was reviewing Alan's code -- that's what I was think of.

2.2 does not use checksum-copy-to-user in TCP RX, csum is either done separate or in hardware.
It does csum-copy-fragment for TX, as well as UDP.

-Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

> > 
> > When you say it reloads it's VM, you mean it reloads the CR3 register?  
> 
> Yes.
> 
> No. In 2.4 you could probably use the on demand lazy vm mechanism ingo described for
> the nfsd processes. In 2.2 it is a bit more tricky, if I remember right lazy mm needed
> quite a few changes.
> 
> But before doing too many changes i would first verify if that is really the problem.

We will never beat NetWare on scaling if this is the case, even in 2.4.  
Andre and my first job will be to create an arch port with MANOS that
disables this and restructures the VM.  

> PTEs are read for aging on memory pressure in vmscan.
> 
> segment register reload happen on interrupt entry, to load the kernel cs/ds (are they that
> costly?). If it was really costly you could probably check in interrupt entry if you're
> already running in kernel space and skip it.

They are.  segment register reloads will trigger the following:

IDT table atomic fetch to verify  (LOCK#) (if triggered by task gate from INTR)
GDT table atomic fetch to verify  (LOCK#)
LDT table atomic fetch to verify  (LOCK#) (if present)
PDE table atomic fetch to verify  (LOCK#)

The process has to verify that the loaded segment descriptor is valid, and
it will fetch from all these tables to do it, with up to 4 (LOCK#) 
assertions occurring invisibly in the hardware underneath (which will
generate 4 non-cacheable memory references, in addition to wrecking 
havoc on the affected L1/L2 cache lines).  Oink.  It only does this 
when you load one, not when you save one, like pushing it on the 
stack.  Since you look at MANOS code, you'll note that in 
CONTEXT.386, I do and add esp, 3 * 4 instead of poppping the segment
registers off the stack if they are in the kernel address space.  

Linux should do the same, if possible as an optimization.


> 
> > 
> 
> 2.2 does not use checksum-copy-to-user in TCP RX, csum is either done separate or in hardware.
> It does csum-copy-fragment for TX, as well as UDP.  

Yes, I know, this is what I was referrring to.  Received are always ugly,
there's always at least one copy into cache for the write, sometimes
more...

Jeff

> 
> -Andi
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 09:39:37AM +0100, Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> > Is there an option to map Linux into a flat address space [...]
> 
> nope, Linux is fundamentally multitasked.
> 
> what you can do to hack around this is to not switch to the idle thread
> after having done work in nfsd. Some simple & stupid thing in schedule:
> 
> 	if (next == idle_task) {
> 		while (nr_running)
> 			barrier();
> 		goto repeat_schedule;
> 	}
> 
> (provided you are testing this on a UP system.) This way we do not destroy
> the TLB cache when we wait a few microseconds for the next network
> interrupt.
> 
> we do this in 2.4 already - ie. nfsd doesnt have to mark itself lazy-MM,
> the idle thread will automatically 'inherit' the MM of nfsd, and is going
> to switch CR3 only if the next process is not nfsd. So you can get an
> apples to apples comparison by using 2.4.

Ingo, I will attempt this, but I doubt seriously it will allow 
Linux to defeat NetWare 5.x on LAN I/O scaling.  ALl protection 
has to go away in all LAN paths for this to happen, and user space
apps set to ring 0.  NetWare 5.cx does support ring 3 applications,
but the model is different than Linux.  I will look at a potential 
compromise between the two with the MANOS /arch merge.   It may 
allow some incarnation of Linux to smoke NetWare 5.x on LAN 
performance.  Doing this would move MARS-NWE and SAMBA into the kernel
without changing a line of code in either...

Jeff


> 
> 	Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Andi Kleen]

On Mon, Oct 30, 2000 at 01:04:34AM -0700, Jeff V. Merkey wrote:
> > > 
> > > When you say it reloads it's VM, you mean it reloads the CR3 register?  
> > 
> > Yes.
> > 
> > No. In 2.4 you could probably use the on demand lazy vm mechanism ingo described for
> > the nfsd processes. In 2.2 it is a bit more tricky, if I remember right lazy mm needed
> > quite a few changes.
> > 
> > But before doing too many changes i would first verify if that is really the problem.
> 
> We will never beat NetWare on scaling if this is the case, even in 2.4.  
> Andre and my first job will be to create an arch port with MANOS that
> disables this and restructures the VM.  

I just guess the end result will be as crash prone as Netware when you install any third
party software ;) 

lazy mm is probably a better path, as long as you stay in kernel threads and a single user mm
it'll never switch VMs.

> 
> > PTEs are read for aging on memory pressure in vmscan.
> > 
> > segment register reload happen on interrupt entry, to load the kernel cs/ds (are they that
> > costly?). If it was really costly you could probably check in interrupt entry if you're
> > already running in kernel space and skip it.
> 
> They are.  segment register reloads will trigger the following:
> 
> IDT table atomic fetch to verify  (LOCK#) (if triggered by task gate from INTR)
> GDT table atomic fetch to verify  (LOCK#)
> LDT table atomic fetch to verify  (LOCK#) (if present)
> PDE table atomic fetch to verify  (LOCK#)
> 
> The process has to verify that the loaded segment descriptor is valid, and
> it will fetch from all these tables to do it, with up to 4 (LOCK#) 
> assertions occurring invisibly in the hardware underneath (which will
> generate 4 non-cacheable memory references, in addition to wrecking 
> havoc on the affected L1/L2 cache lines).  Oink.  It only does this 
> when you load one, not when you save one, like pushing it on the 
> stack.  Since you look at MANOS code, you'll note that in 
> CONTEXT.386, I do and add esp, 3 * 4 instead of poppping the segment
> registers off the stack if they are in the kernel address space.  
> 
> Linux should do the same, if possible as an optimization.

Interesting. You could do easily the same in Linux by changing (in 2.2) the SAVE_ALL macro
in arch/i386/kernel/irq.h and doing the same in ret_from_intr's RESTORE_ALL macro (after
changing it to a RESTORE_ALL_INT or you'll break system calls) 

Actually I don't even know why irq.h's SAVE_ALL even loads __KERNEL_CS, it should be already
set by the interrupt gate. __KERNEL_DS probably needs to be still loaded, but only when
you came from user space.

-Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Andi Kleen wrote:

> One problem in Linux 2.2 is that kernel threads reload their VM on
> context switch (that would include the nfsd thread), this should be
> fixed in 2.4 with lazy mm. Hmm actually it should be only fixed for
> true kernel threads that have been started with kernel_thread(), the
> "pseudo kernel threads" like nfsd uses probably do not get that
> optimization because they don't set their MM to init_mm.

yes, but for this there is an explicit mechanizm to lazy-MM during lengthy
system calls, an example is in buffer.c:

                user_mm = start_lazy_tlb();
                error = sync_old_buffers();
                end_lazy_tlb(user_mm);

> > to get disproportiantely higher in Linux than NetWare 5.x and when it hits
> > 60% of total clock cycles, Linux starts dropping off.  NetWare 5.x is 1/8 
> 
> I think that can be explained by the copying.

yes. Constant copying contaminates the L1/L2 caches and creates dirty
cachelines all around the place. Fixed in 2.4 + TUX ;-)

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> Is there an option to map Linux into a flat address space [...]

nope, Linux is fundamentally multitasked.

what you can do to hack around this is to not switch to the idle thread
after having done work in nfsd. Some simple & stupid thing in schedule:

	if (next == idle_task) {
		while (nr_running)
			barrier();
		goto repeat_schedule;
	}

(provided you are testing this on a UP system.) This way we do not destroy
the TLB cache when we wait a few microseconds for the next network
interrupt.

we do this in 2.4 already - ie. nfsd doesnt have to mark itself lazy-MM,
the idle thread will automatically 'inherit' the MM of nfsd, and is going
to switch CR3 only if the next process is not nfsd. So you can get an
apples to apples comparison by using 2.4.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 10:52:08AM +0100, Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> > [...] All protection has to go away in all LAN paths for this to
> > happen, and user space apps set to ring 0. [...]
> 
> i found that this is not a requirement for good network scalability. We do
> not do a syscall for every packet, so the cost evens out. Sure, it does
> not hurt to not eat ~1 microsecond per system-call, but it causes no
> overhead or scalability limit otherwise. In the TUX webserver we have
> user-space modules doing context-switch-less webserving, and it scales
> quite well, and is generic.
> 
> 	Ingo

No argument here, but the overhead of reloading CR3 period will kill
performance.  2.4 does not beat NetWare, BTW, it gets a little further,
but still hits the wall, and Netware keeps going strong, and scales 
up to 5000 users on file and print, with a SINGLE processor at less <
40% utilization.  Linux hits the wall at a few hundred file and print
users.  It's the overhead of CR3 switching at all that is causing this
and the heavy usage of Intel's protection model.

For example, if you put a MOV EAX, CR3;  MOV CR3, EAX; in a context switching
path, on a PPro 200, you can do about 35,000 context switches/second
(I did this in NetWare 4.x in 1994) with the same code without the 
CR3 reload, I could do over 2,000,000 context switches/second without
the CR3 load.  INVL Page[0] in the same path is less heavy, but still
reduced context switches to 135,000/second.  It's the CR3 activity period
that kills performance.  There's also the use of segment registers
all over the place to copy from kernel to user and user to kernel 
space memory. Having the fast paths you mention does help a lot,
but it's the fact that this goes on at all that will make it tough 
to walk into a NetWare shop with Linux and rip out NetWare servers
and replace them unless we look at a NetWare vs. NetLinux (that's 
what we call it! a NetWare-like Linux platform).  

If we had this, MS would be running for the trenches.  Coupling the 
internet application base of Linux with the speed of NetWare would
be Mr. Gate's worst nightmare.  They've been trying to kill off NetWare
for almost 15 years, and it's still out there and they are still 
slower, and everyone knows it.  It's not Linux or NT that's killing 
off NetWare, right now, it's the incompetent management in San Jose
who are trying to rape Novell's rich bank accounts 
(Schmidt and Sonsini and friends), and the fact they have no IA64
NetWare (they would be shipping it not if I were still there ....).

It could be as easy as a compile option in .config (NETWARE_SERVER_MODE [Y/N])
with WTD (which is a more sophistacted method for doing lazy VM and 
controlling context switching activity for fast path I/O) and 
mapping of te Linux address space as linear -- you can still have 
protection with this model, just some apps would be able to load
in the kernel address space, and run at ring 0).

:-)

Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 11:13:58AM +0100, Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> > No argument here, but the overhead of reloading CR3 period will kill
> > performance. [...]
> 
> 2.4 does not reload CR3, unless you are using multiple user-space
> processes.
> 
> > 2.4 does not beat NetWare, BTW, it gets a little further, but still
> > hits the wall, [...]
> 
> as i told you in the previous mail, the main overhead is not CR3, it's the
> copying & dirtying of all data, and the subsequent DMA-initiated dirty
> cacheline writeback. I can serve 100 MB/sec web content with 2.4 & TUX
> just fine - it relies on a zero-copying infrastructure.
> 
> 	Ingo


Great Ingo, you've got the web server covered.  What about file and print.  
I think this is great, but most web servers are connected to a T1 or T3 
line, and all the fancy optimization means absolutely squat since about 
99.999999% of the planet has a max baandwidth of a T1, ADSL, or T3 Line,
this is a far cry from Gigabit ethernet, or even 100Mbit ethernet.  

How many users can you put on the web server?  Web servers are also 
read only data, the easiest of all LAN cases to deal with.  It's 
incoming writes that present all the tough problems, as Andi Klein
wisely observed.  Not to knock Tux, I think it's great, but it does
not solve the file and print scaling problem, no does it?

Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 11:27:04AM +0100, Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> > For example, if you put a MOV EAX, CR3;  MOV CR3, EAX; in a context
> > switching path, on a PPro 200, you can do about 35,000 context
> > switches/second
> 
> in 2.4 & Xeons we can do more than 100,000 context switches/second, and
> that is more than enough. But the point is: network IO performance does
> not depend on context switching speed too much. Also, in Linux we are
> using global pages which makes kernel-space TLBs persistent even across
> CR3 flushes.

This is putrid.  NetWare does 353,00,000/second on a Xenon, pumping out
gobs of packets in between them.  MANOS does 857,000,000/second.  This 
is terrible.  No wonder it's so f_cking slow!!!

> 
> > [...] There's also the use of segment registers all over the place to
> > copy from kernel to user and user to kernel space memory. [...]
> 
> we do not use the fs segment register for user-space copies anymore,
> neither in 2.2, nor in 2.4. You must be reading old books and probably
> forgot to cross-check with the kernel source? :-)


ds: and es: are both used in copy-to-user and copy-from-user and they get 
reloaded.


> 
> > [...] Having the fast paths you mention does help a lot, but it's the
> > fact that this goes on at all that will make it tough to walk into a
> > NetWare shop with Linux and rip out NetWare servers and replace them
> > unless we look at a NetWare vs. NetLinux (that's what we call it! a
> > NetWare-like Linux platform).
> 
> the worst thing you can do is to mis-identify performance problems and
> spend braincells on the wrong problem. The problems limiting Linux network
> scalability have been identified during the last 12 months by a small
> team, and solved in TUX. TUX is a fileserver, it shouldnt be alot of work
> to enable it for (TCP-only?) netware serving. It's *done*, Jeff, it's not
> a hypotetical thing, it's here, it works and it performs.
> 

NetWare is here too, and it handles 5000+ file and print users, Linux does not.
Let's fix it.  I know why NetWare is fast.  Let's apply some of the same 
principles and see what happens.  Love to have you involved.

> 	Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 11:41:35AM +0100, Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> > [...] you've got the web server covered.  What about file and print.
> 
> a web server, as you probably know, is a read-mostly fileserver that
> serves files via the HTTP protocol. The rest is only protocol fluff.
> 
> > I think this is great, but most web servers are connected to a T1 or
> > T3 line, and all the fancy optimization means absolutely squat since
> > about 99.999999% of the planet has a max baandwidth of a T1, ADSL, or
> > T3 Line, this is a far cry from Gigabit ethernet, or even 100Mbit
> > ethernet.
> 
> Your argument is curious - first you cry for performance, then you say
> 'nobody wants that much bandwidth'. Of course, if the network is bandwidth
> limited then we cannot scale above that bandwidth. But thats not the
> point. The point is to put 10 cards into a server and still being able to
> saturate them. The point is also to spend less cycles on saturating
> available bandwidth. The point is also to not flush the L1 just because
> someone requested a 10K webpage.

It's not curious, it's not about bandwidth, it's about latency, and getting
packets in and out of the server as fast as possible, and ahead of everything
else.  Cache affinity and all the Tux stuff is a great piece of work.  Let's
talk about file and print, that's the present problem, not how to 
pump out read-only data from a web server.


> 
> > How many users can you put on the web server? [...]
> 
> tens of thousands, on a single CPU. Can probably handle over 100 thousand
> users as well, with IP aliasing so the socket space is spread out.
> 
> > Web servers are also read only data, the easiest of all LAN cases to
> > deal with. It's incoming writes that present all the tough problems,
> 
> reads dominate writes in almost all workloads, thats common wisdom. Why
> write if nobody reads the data? And while web servers are mostly read only
> data, they can write data as well, see POST and PUT. The fact that
> incoming writes are hard should not let you distract from the fact that
> reads are also extremely important.
> 
> 	Ingo


Web servers don't do writes, unless a CGI script is running somewhere
or some Java or Perl or something, then this stuff goes through a 
wrapper, which is slow, or did I miss something.  

Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 11:44:26AM +0100, Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> > This is putrid. NetWare does 353,00,000/second on a Xenon, pumping out
> > gobs of packets in between them. MANOS does 857,000,000/second. This
> > is terrible. No wonder it's so f_cking slow!!!

And please check your numbers, 857 million
> context switches per second means that on a 1 GHZ CPU you do one context
> switch per 1.16 clock cycles. Wow!

Excuse me, 857,000,000 instructions executed and 460,000,000 context switches
a second -- on a PII system at 350 Mhz.  It's due to AGI optimization.  
Download MANOS and verify for yourself, it has a built in EMON in monitor.
After I complete the port, not even NetWare will be able to touch it.

Your Tux web server will also run on it, at significantly increased 
performance.  

Jeff

> 
> 	Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 11:50:24AM +0100, Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> > ds: and es: are both used in copy-to-user and copy-from-user and they
> > get reloaded.
> 
> And they all share the same segment descriptor. Whats your point? ES is
> the default target segment for string operations. DS is the default data
> segment. Have you ever profiled how many cycles it takes to do a "mov
> __KERNEL_DS, %es" in entry.S, before making your (ridiculous) claim? I
> have.
> 

No.  I used a hardware analyzer to show me how many LOCK# assertions it does
invisible to your software tools underneath.  Try using EMON to profile,
it gives hardware numbers and let's you watch the cache controllers 
issue non-cacheable memory references to fetch the descriptors.   

Jeff


> 	Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 11:56:06AM +0100, Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> > > reads dominate writes in almost all workloads, thats common wisdom. Why
> > > write if nobody reads the data? And while web servers are mostly read only
> > > data, they can write data as well, see POST and PUT. The fact that
> > > incoming writes are hard should not let you distract from the fact that
> > > reads are also extremely important.
> >
> > Web servers don't do writes, unless a CGI script is running somewhere
> > or some Java or Perl or something, then this stuff goes through a
> > wrapper, which is slow, or did I miss something.
> 
> yes, you missed TUX modules.
> 


Great.  I can load a TUX module and use it with my T1 line.  If I could
spare an extra $100,000/month, perhaps I can lease an SDM-172 or TAT-8, or
even an OC-172 then I would be able to take advantage of it in the real
world.

Jeff


> 	Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> [...] All protection has to go away in all LAN paths for this to
> happen, and user space apps set to ring 0. [...]

i found that this is not a requirement for good network scalability. We do
not do a syscall for every packet, so the cost evens out. Sure, it does
not hurt to not eat ~1 microsecond per system-call, but it causes no
overhead or scalability limit otherwise. In the TUX webserver we have
user-space modules doing context-switch-less webserving, and it scales
quite well, and is generic.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 12:01:08PM +0100, Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> > > And please check your numbers, 857 million
> > > context switches per second means that on a 1 GHZ CPU you do one context
> > > switch per 1.16 clock cycles. Wow!
> > 
> > Excuse me, 857,000,000 instructions executed and 460,000,000 context switches
> > a second -- on a PII system at 350 Mhz. [...]
> 
> so it does 1.3 context switches per clock cycle? Wow! And i can type
> 100000000000000000000 characters a second, just measured it. Really!

Go download it and try it, then come back with that smirk wiped off your face.
I'll enjoy it.....

:-)

> 
> > Your Tux web server will also run on it, at significantly increased
> > performance.
> 
> as i told you in the previous mails, TUX does not depend on schedule()
> performance. schedule() cost does not even show up in the top 20 entries
> of the profiler.


And as I told, you, your code has nothing to do with it, it's the fact it 
goes on at all.  Ingo, go get a copy of NetWare 3.12 (I'll even send you
one -- I've got extra licensed copies), install it, put a load of 5000
connections on it, with 4 adapters.  Dual boot Linux on it, and attempt 
the same with SAMBA or MARS-NWE, and watch it oink.  

Go do it.  What's your address to I can ship you the CD's for 3.12.  Then come
back and tell me how TUX is going to solve the File and Print performance
issues in Linux.

:-)

Jeff

 
> 
> 	Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 12:04:43PM +0100, Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> > It's not curious, it's not about bandwidth, it's about latency, and
> > getting packets in and out of the server as fast as possible, and
> > ahead of everything else. [...]
> 
> TUX prepares a HTTP reply in about 30 microseconds (plus network latency),
> good enough? Network latency is the limit, even on gigabit - not to talk
> about T1 lines.

Great.  Now how do we get the smae numbers on SAMBA and MARS-NWE?   THat's 
the question, not whether your baby TUX is pretty.  I already said it
was pretty, focus on the other issue.

Jeff

> 
> 	Ingo
> 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 12:12:44PM +0100, Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> > > > Excuse me, 857,000,000 instructions executed and 460,000,000 context switches
> > > > a second -- on a PII system at 350 Mhz. [...]
> 
> > Go download it and try it, then come back with that smirk wiped off
> > your face. I'll enjoy it.....
> 
> so in 0.53 clock cycles you are implementing things like address space
> separation, process priorities, fairness and other essential scheduling
> features? Truly awesome ...

Ingo,  This original thread was regarding Linux vs. NetWare 5.x performance
metrics and responses from Linux folks about how to affect and 
improve them, not a diatribe on the features of TUX.  

I wrote the kernel in NetWare 4.x that later became 5.x.  To date, my
NOS kernel has grossed over 8 billion dollars.  This is more money 
than all the Linux companies have made combined in their entire 
history, on your work, Linus's work and everyone else's.  I don't have 
anything to prove to myself, or anyone else, which is why I do as I 
please in life, and no one's comments or snipes dissuade me from moving
forward.  I also am not "someone's employee".  

If you have some ideas on how to improve file and print or help me
get a linux incarnation that can stomp NetWare, I'd love to hear
your ideas.  I think TUX is great, BTW.  Otherwise, end-of-line...

:-)

Jeff


> 
> 	Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

On Mon, Oct 30, 2000 at 12:13:52PM +0100, Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> > > TUX prepares a HTTP reply in about 30 microseconds (plus network latency),
> > > good enough? Network latency is the limit, even on gigabit - not to talk
> > > about T1 lines.
> > 
> > Great.  Now how do we get the smae numbers on SAMBA and MARS-NWE? [...]
> 
> simple, write a TUX protocol module for it. FTP protocol module is on its
> way. Stay tuned.
>  
> 	Ingo


I do not believe this approach will allow Linux to match NetWare's 
file and print performance, but I am willing to give it a whirl.  Where
is the TUX module for MARS.  Let's start with this one.  What help
would you require.  You understand these TUX modules, so you should
take the lead.

I am listening.  Instruct me....


Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> No argument here, but the overhead of reloading CR3 period will kill
> performance. [...]

2.4 does not reload CR3, unless you are using multiple user-space
processes.

> 2.4 does not beat NetWare, BTW, it gets a little further, but still
> hits the wall, [...]

as i told you in the previous mail, the main overhead is not CR3, it's the
copying & dirtying of all data, and the subsequent DMA-initiated dirty
cacheline writeback. I can serve 100 MB/sec web content with 2.4 & TUX
just fine - it relies on a zero-copying infrastructure.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> For example, if you put a MOV EAX, CR3;  MOV CR3, EAX; in a context
> switching path, on a PPro 200, you can do about 35,000 context
> switches/second

in 2.4 & Xeons we can do more than 100,000 context switches/second, and
that is more than enough. But the point is: network IO performance does
not depend on context switching speed too much. Also, in Linux we are
using global pages which makes kernel-space TLBs persistent even across
CR3 flushes.

> [...] There's also the use of segment registers all over the place to
> copy from kernel to user and user to kernel space memory. [...]

we do not use the fs segment register for user-space copies anymore,
neither in 2.2, nor in 2.4. You must be reading old books and probably
forgot to cross-check with the kernel source? :-)

> [...] Having the fast paths you mention does help a lot, but it's the
> fact that this goes on at all that will make it tough to walk into a
> NetWare shop with Linux and rip out NetWare servers and replace them
> unless we look at a NetWare vs. NetLinux (that's what we call it! a
> NetWare-like Linux platform).

the worst thing you can do is to mis-identify performance problems and
spend braincells on the wrong problem. The problems limiting Linux network
scalability have been identified during the last 12 months by a small
team, and solved in TUX. TUX is a fileserver, it shouldnt be alot of work
to enable it for (TCP-only?) netware serving. It's *done*, Jeff, it's not
a hypotetical thing, it's here, it works and it performs.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> [...] you've got the web server covered.  What about file and print.

a web server, as you probably know, is a read-mostly fileserver that
serves files via the HTTP protocol. The rest is only protocol fluff.

> I think this is great, but most web servers are connected to a T1 or
> T3 line, and all the fancy optimization means absolutely squat since
> about 99.999999% of the planet has a max baandwidth of a T1, ADSL, or
> T3 Line, this is a far cry from Gigabit ethernet, or even 100Mbit
> ethernet.

Your argument is curious - first you cry for performance, then you say
'nobody wants that much bandwidth'. Of course, if the network is bandwidth
limited then we cannot scale above that bandwidth. But thats not the
point. The point is to put 10 cards into a server and still being able to
saturate them. The point is also to spend less cycles on saturating
available bandwidth. The point is also to not flush the L1 just because
someone requested a 10K webpage.

> How many users can you put on the web server? [...]

tens of thousands, on a single CPU. Can probably handle over 100 thousand
users as well, with IP aliasing so the socket space is spread out.

> Web servers are also read only data, the easiest of all LAN cases to
> deal with. It's incoming writes that present all the tough problems,

reads dominate writes in almost all workloads, thats common wisdom. Why
write if nobody reads the data? And while web servers are mostly read only
data, they can write data as well, see POST and PUT. The fact that
incoming writes are hard should not let you distract from the fact that
reads are also extremely important.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> This is putrid. NetWare does 353,00,000/second on a Xenon, pumping out
> gobs of packets in between them. MANOS does 857,000,000/second. This
> is terrible. No wonder it's so f_cking slow!!!

(no need to get emotional.) And please check your numbers, 857 million
context switches per second means that on a 1 GHZ CPU you do one context
switch per 1.16 clock cycles. Wow!

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> ds: and es: are both used in copy-to-user and copy-from-user and they
> get reloaded.

And they all share the same segment descriptor. Whats your point? ES is
the default target segment for string operations. DS is the default data
segment. Have you ever profiled how many cycles it takes to do a "mov
__KERNEL_DS, %es" in entry.S, before making your (ridiculous) claim? I
have.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> > reads dominate writes in almost all workloads, thats common wisdom. Why
> > write if nobody reads the data? And while web servers are mostly read only
> > data, they can write data as well, see POST and PUT. The fact that
> > incoming writes are hard should not let you distract from the fact that
> > reads are also extremely important.
>
> Web servers don't do writes, unless a CGI script is running somewhere
> or some Java or Perl or something, then this stuff goes through a
> wrapper, which is slow, or did I miss something.

yes, you missed TUX modules.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[john slee]

On Mon, Oct 30, 2000 at 03:06:25AM -0700, Jeff V. Merkey wrote:
> Ingo,  This original thread was regarding Linux vs. NetWare 5.x performance
> metrics and responses from Linux folks about how to affect and 
> improve them, not a diatribe on the features of TUX.

while beating netware in certain areas is certainly a noble goal,
it is far from the only one.

[snip]

> If you have some ideas on how to improve file and print or help me
> get a linux incarnation that can stomp NetWare, I'd love to hear
> your ideas.  I think TUX is great, BTW.  Otherwise, end-of-line...

if netware users are happy with netware, i can't see why they'd want to
switch to linux or a linux-netware-ish equivalent while netware are
still around to provide support.  "ain't broke, don't fix," etc.  

j.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> > And please check your numbers, 857 million
> > context switches per second means that on a 1 GHZ CPU you do one context
> > switch per 1.16 clock cycles. Wow!
> 
> Excuse me, 857,000,000 instructions executed and 460,000,000 context switches
> a second -- on a PII system at 350 Mhz. [...]

so it does 1.3 context switches per clock cycle? Wow! And i can type
100000000000000000000 characters a second, just measured it. Really!

> Your Tux web server will also run on it, at significantly increased
> performance.

as i told you in the previous mails, TUX does not depend on schedule()
performance. schedule() cost does not even show up in the top 20 entries
of the profiler.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> It's not curious, it's not about bandwidth, it's about latency, and
> getting packets in and out of the server as fast as possible, and
> ahead of everything else. [...]

TUX prepares a HTTP reply in about 30 microseconds (plus network latency),
good enough? Network latency is the limit, even on gigabit - not to talk
about T1 lines.

	Ingo


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [patch] kernel/module.c (plus gratuitous rant)

[Peter Samuelson]

  [rth]
> > Which was a nice idea, but it doesn't actually work.  Changes
> > in spec file format between versions makes this fall over.

[Dominik Kubla]
> Wow. So much for reading the manual... well, that's considered
> cheating anyway, isn't it?

I know this was true at one time -- egcs couldn't read 2.7 spec files,
or something like that.  (I remember at the time thinking "so much for
the great and glorious '-V' theory".)

But I think it's since been fixed:

  $ gcc -v
  Reading specs from /usr/lib/gcc-lib/i386-linux/2.95.2/specs
  gcc version 2.95.2 20000220 (Debian GNU/Linux)
  $ gcc -V2.7.2.3 -v
  Reading specs from /usr/lib/gcc-lib/i386-linux/2.7.2.3/specs
  gcc driver version 2.95.2 20000220 (Debian GNU/Linux) executing gcc version 2.7.2.3

Is there more subtle breakage?

Peter
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [patch] kernel/module.c (plus gratuitous rant)

[Peter Samuelson]

[Rusty]
> > CC=gcc-2723 make 2.0 kernel
> > CC=gcc-2723 make 2.2 kernel
> > CC=egcs make 2.4 kernel
> 
> No, environment doesn't override make variables by default.  This
> works on any shell:
> 
> 	make CC=egcs <targets>

If you're going to get pedantic, that won't work either -- since the
makefiles in kernels 2.0 and 2.2 expect $(CC) to include some compiler
flags.  This was fixed somewhere in 2.3.3x.

Peter
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> > > Excuse me, 857,000,000 instructions executed and 460,000,000 context switches
> > > a second -- on a PII system at 350 Mhz. [...]

> Go download it and try it, then come back with that smirk wiped off
> your face. I'll enjoy it.....

so in 0.53 clock cycles you are implementing things like address space
separation, process priorities, fairness and other essential scheduling
features? Truly awesome ...

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> > TUX prepares a HTTP reply in about 30 microseconds (plus network latency),
> > good enough? Network latency is the limit, even on gigabit - not to talk
> > about T1 lines.
> 
> Great.  Now how do we get the smae numbers on SAMBA and MARS-NWE? [...]

simple, write a TUX protocol module for it. FTP protocol module is on its
way. Stay tuned.
 
	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Recommended compiler? - Re: [patch] kernel/module.c (plus gratuitous rant)

[Linux Kernel Developer]

So which is the recommended compiler for each kernel version 2.2.x,
2.4.x(pre?) nowadays?  I've pretty much kept gcc 2.7.2.3 around just for
compiling the kernel however now I hear you need egcs to compile 2.4?  I
don't mind keeping 2.7.2.3 around in its own installation directory just for
the purpose of doing kernel work however from a previous post I've now got
the impression that egcs has become the recommended compiler?  If I'm going
to keep a secondary compiler around (outside of gcc 2.95.2 which I still
hear is no good for kernel compiles) just for kernel work I'd prefer to use
my disk space on the recommended one.

>
> [Rusty]
> > > CC=gcc-2723 make 2.0 kernel
> > > CC=gcc-2723 make 2.2 kernel
> > > CC=egcs make 2.4 kernel
> >
> > No, environment doesn't override make variables by default.  This
> > works on any shell:
> >
> > make CC=egcs <targets>
>
> If you're going to get pedantic, that won't work either -- since the
> makefiles in kernels 2.0 and 2.2 expect $(CC) to include some compiler
> flags.  This was fixed somewhere in 2.3.3x.
>
> Peter
> -


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> Ingo, This original thread was regarding Linux vs. NetWare 5.x
> performance metrics and responses from Linux folks about how to affect
> and improve them, not a diatribe on the features of TUX.

oh, i believe you misunderstand. TUX itself is quite simple. But it
extended the Linux TCP stack and scalability to new levels (which has
nothing to do with TUX itself, it's the scalability of the Linux
networking stack that evolved gradually over the past 10 years - we spend
95% of the time outside of TUX!). And if you claim that Linux needs this
and that for scalability, then i'd like to point you humbly towards those
existing, generic and well-performing results. TUX is just a 5% 'HTTP
protocol fluff' around the generic stuff.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Recommended compiler? - Re: [patch] kernel/module.c (plus gratuitous rant)

[Peter Samuelson]

> So which is the recommended compiler for each kernel version 2.2.x,
> 2.4.x(pre?) nowadays?

* 2.91.66 aka egcs 1.1.2.  It has been officially blessed for 2.4 and
  has been given an informal thumbs-up by Alan for 2.2.  (It does NOT
  work for 2.0, if you still care about that.)

* 2.7.2.3 works for 2.2 (and 2.0) but NOT for 2.4.

* 2.95.2 seems to work with both 2.2 and 2.4 (no known bugs, AFAIK) and
  many of us use it, but it is a little riskier than egcs.

* Red Hat "2.96" or CVS 2.97 will probably break any known kernel.

Peter
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: / on ramfs, possible?

[Alan Cox]

> I want my / to be a ramfs filesystem. I intend to populate it from an 
> initrd image, and then remount / as the ramfs filesystem. Is that at 
> all possible? The way I see it the kernel requires / on a device 
> (major,minor) or nfs.
> 
> Am I out of luck using ramfs as /? If it's easy to fix, how do I fix it?

Firstly make sure you get the patches that make ramfs work if they arent all
yet applied, then build your initrd that populates it on boot. Now you can
make use of the pivot_root() syscall (you may need to generate your own 
syscall wrapper for that one as its very new). That lets you flip your root
and initrd around

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Alan Cox]

> We will never beat NetWare on scaling if this is the case, even in 2.4.  
> Andre and my first job will be to create an arch port with MANOS that
> disables this and restructures the VM.  

In the 2.4 case if you are just running NFS daemons then there are no tlb
reloads going on at all. Whats murdering you is mostly memory copies. I would
suspect if you rerun the profiles on a box with a much lower memory bandwidth
that the effect will be even more visible
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Andi Kleen]

On Mon, Oct 30, 2000 at 12:47:10PM +0000, Alan Cox wrote:
> > We will never beat NetWare on scaling if this is the case, even in 2.4.  
> > Andre and my first job will be to create an arch port with MANOS that
> > disables this and restructures the VM.  
> 
> In the 2.4 case if you are just running NFS daemons then there are no tlb
> reloads going on at all. Whats murdering you is mostly memory copies. I would
> suspect if you rerun the profiles on a box with a much lower memory bandwidth
> that the effect will be even more visible

I don't think that's true. As far as I can see the nfsd processes do not do the
lazy VM magic (but it would be reasonably easy to add) 


-Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Alan Cox]

> one -- I've got extra licensed copies), install it, put a load of 5000
> connections on it, with 4 adapters.  Dual boot Linux on it, and attempt 
> the same with SAMBA or MARS-NWE, and watch it oink.  

SAMBA and Mars-nwe are running user space thats why. They have flexibility,
protection and can run unpriviledged. If you want to put mars-nwe in the kernel
then it will certainly be interesting

> back and tell me how TUX is going to solve the File and Print performance
> issues in Linux.

There's an http file system around 8)

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Recommended compiler? - Re: [patch] kernel/module.c (plus gratuitous rant)

[Martin Dalecki]

Peter Samuelson wrote:
> 
> > So which is the recommended compiler for each kernel version 2.2.x,
> > 2.4.x(pre?) nowadays?
> 
> * 2.91.66 aka egcs 1.1.2.  It has been officially blessed for 2.4 and
>   has been given an informal thumbs-up by Alan for 2.2.  (It does NOT
>   work for 2.0, if you still care about that.)
> 
> * 2.7.2.3 works for 2.2 (and 2.0) but NOT for 2.4.
> 
> * 2.95.2 seems to work with both 2.2 and 2.4 (no known bugs, AFAIK) and
>   many of us use it, but it is a little riskier than egcs.
> 
> * Red Hat "2.96" or CVS 2.97 will probably break any known kernel.

Works fine for me and 2.4.0-test10-pre5... however there are tons of
preprocessor warnings in some drivers.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Andrea Arcangeli]

On Mon, Oct 30, 2000 at 12:13:52PM +0100, Ingo Molnar wrote:
> simple, write a TUX protocol module for it. FTP protocol module is on its
> way. Stay tuned.

TUX modules are kernel modules (I mean you have to write kernel space code for
doing TUX ftp). Don't you agree that zero-copy sendfile like ftp serving would
be able to perform equally well too? I mean: isn't better to spend the efforts
to make an userspace API to run fast instead of moving every network
functionality that needs high performance completly in kernel? People may need
to write high performance network code for custom protocols, this way they will
end creating kernel modules with system-crashing bugs, memory leaks and kernel
buffer overflows (chroot+nobody+logging won't work anymore). (plus they will
get into pain while debugging)

It's obvious kernel code runs faster and you can also do assumptions about
scheduler and current CPU that you can't do in userspace, but is that so
relevant in term of ftp server numbers compared to only skipping the memory
copies?

About the TUX cgi module I had a fast look and I noticed cgis run by tux
executes two clones(2) and one exec(2) and then they have to pay the startup of
the interpreters for each cgi request. So at this stage of tux I guess that for
perl/php pages tux would better redirected them to an apache.  Maybe php and
perl tux (kernel) modules are in your todo list? (I think they would be a bad
idea though) One other way to handle efficiently the interpreted-cgi load
without redirecting the cgi request to a full apache is to have a background
php/perl interpreter listening to new cgis in input and filling the tux pipe in
output.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Alan,

I've been studying Linux for the past two years the same way a diamond
cutter studies a 
prized and immensely valuable raw stone.   I think I am nearing the
point I know where to
strike and I can cleave it  into something Novell's installed base would
like and could move
forward with.  The easiest path I see for me to create a Linux NetWare
hybrid is for you 
guys to get a 2.2.X (not 2.4.X -- not just yet) tree that runs in a flat
memory space,
then I can slip in our optimizations and we will be able to run all
existing code in kernel
that's trusted.  The changes needed to do this involve more than just
the /arch area and 
the asm macro includes, though this will cover about 80% of it, but the
loader would need to 
be able to create address domains around aps that choose to run
protected.  

If I attempt these changes without proper guidance, two things will
happen.  1).  people who
own some of this code will not be aware of all the aspects of the
changes, and may not 
longer be able to support it, and 2).  it will fork away from the tree
and become 
unsupportable and diverge over time.  This is the reason I have resisted
making any 
kernel changes myself other than code I write for Linux I own, and I
have always 
solicitied other folks to make the patches.  This is about to change
relative to this
project.

The first step I need is to be able to load Linux as a ring 0 OS in
linear address space 
with all protection disabled.  Paging can still go on, just no separate
CR3 reloads between
context switches.   profiling Ring 0 Linux vs. NetWare will give me an
excellent idea of where 
the optimizations will need to be inserted.  A straight MARS-NWE port to
kernel would just
happen, since we would be able to just load in kernel space and run it
with no code 
changes.  

:-)

Jeff

Alan Cox wrote:
> 
> > one -- I've got extra licensed copies), install it, put a load of 5000
> > connections on it, with 4 adapters.  Dual boot Linux on it, and attempt
> > the same with SAMBA or MARS-NWE, and watch it oink.
> 
> SAMBA and Mars-nwe are running user space thats why. They have flexibility,
> protection and can run unpriviledged. If you want to put mars-nwe in the kernel
> then it will certainly be interesting
> 
> > back and tell me how TUX is going to solve the File and Print performance
> > issues in Linux.
> 
> There's an http file system around 8)
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Chris Evans]

On Mon, 30 Oct 2000, Andrea Arcangeli wrote:

> functionality that needs high performance completly in kernel? People
> may need to write high performance network code for custom protocols,
> this way they will end creating kernel modules with system-crashing
> bugs, memory leaks and kernel buffer overflows (chroot+nobody+logging
> won't work anymore). (plus they will get into pain while debugging)

I'm glad _someone_ is connected to reality with regards the security
implications of throwing loads of servers into kernel space.

Cheers
Chris

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Andrea Arcangeli wrote:
> 
> On Mon, Oct 30, 2000 at 12:13:52PM +0100, Ingo Molnar wrote:
> > simple, write a TUX protocol module for it. FTP protocol module is on its
> > way. Stay tuned.
> 
> TUX modules are kernel modules (I mean you have to write kernel space code for
> doing TUX ftp). Don't you agree that zero-copy sendfile like ftp serving would
> be able to perform equally well too? I mean: isn't better to spend the efforts
> to make an userspace API to run fast instead of moving every network
> functionality that needs high performance completly in kernel? People may need
> to write high performance network code for custom protocols, this way they will
> end creating kernel modules with system-crashing bugs, memory leaks and kernel
> buffer overflows (chroot+nobody+logging won't work anymore). (plus they will
> get into pain while debugging)

Ingo's helping me get the info together on this for putting a MARS-NWE
tux module in 
the kernel.  He had to go do some things this week he told me before he
would be ready
to look at it.  He did point me over to the info, and I agreed we would
attempt to 
implement it as something to look at.  If it performs well enough, I
will have 
something reasonable to send out to Novell Resellers (CNEs) and
Cutomers.

Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Chris Evans wrote:
> 
> On Mon, 30 Oct 2000, Andrea Arcangeli wrote:
> 
> > functionality that needs high performance completly in kernel? People
> > may need to write high performance network code for custom protocols,
> > this way they will end creating kernel modules with system-crashing
> > bugs, memory leaks and kernel buffer overflows (chroot+nobody+logging
> > won't work anymore). (plus they will get into pain while debugging)
> 
> I'm glad _someone_ is connected to reality with regards the security
> implications of throwing loads of servers into kernel space.
> 

If we implement a ring 0 Linux, all of this will remain intact with the
need to 
port modules into the kernel at all.  

Jeff

> Cheers
> Chris
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

john slee wrote:
> 
> On Mon, Oct 30, 2000 at 03:06:25AM -0700, Jeff V. Merkey wrote:
> > Ingo,  This original thread was regarding Linux vs. NetWare 5.x performance
> > metrics and responses from Linux folks about how to affect and
> > improve them, not a diatribe on the features of TUX.
> 
> while beating netware in certain areas is certainly a noble goal,
> it is far from the only one.

You're up on that higher plane again.  Send me some 'shrooms, please. 

> 
> [snip]
> 
> > If you have some ideas on how to improve file and print or help me
> > get a linux incarnation that can stomp NetWare, I'd love to hear
> > your ideas.  I think TUX is great, BTW.  Otherwise, end-of-line...
> 
> if netware users are happy with netware, i can't see why they'd want to
> switch to linux or a linux-netware-ish equivalent while netware are
> still around to provide support.  "ain't broke, don't fix," etc.

Linux has great internet connectivity, and a real application layer with
lots of 
apps.  NetWare customers would love it.  I am one and I love it already.

Jeff

> 
> j.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Andrea Arcangeli]

On Mon, Oct 30, 2000 at 11:01:25AM -0700, Jeff V. Merkey wrote:
> If we implement a ring 0 Linux, all of this will remain intact with the
> need to 
> port modules into the kernel at all.  

I don't understand what you mean sorry, could you elaborate?

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Alan Cox]

> context switches.   profiling Ring 0 Linux vs. NetWare will give me an
> excellent idea of where 
> the optimizations will need to be inserted.  A straight MARS-NWE port to
> kernel would just
> happen, since we would be able to just load in kernel space and run it
> with no code 
> changes.  

There are one bunch of people running Linux on a flat memory space with no
protection although their goal was to make Linux run on mmuless embedded
hardware.

See www.uclinux.org; the uclinux guys started a 2.4 port recently. Basically
the idea is to have a mm-nommu/ directory which implements a mostly compatible
replacement for the mm layer (obviously stuff like mmap dont work without an
mmu and fork is odd), and a set of binary loaders to load flat binaries with
relocations.

That I think is the project that overlaps ..

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [patch] kernel/module.c (plus gratuitous rant)

[Richard Henderson]

On Mon, Oct 30, 2000 at 05:05:43AM -0600, Peter Samuelson wrote:
> But I think it's since been fixed:

No.

> Is there more subtle breakage?

Yes.


r~
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Dan Hollis]

On Mon, 30 Oct 2000, Andrea Arcangeli wrote:
> TUX modules are kernel modules (I mean you have to write kernel space code for
> doing TUX ftp). Don't you agree that zero-copy sendfile like ftp serving would
> be able to perform equally well too?

For this to bw useful for ftp we need a sendfile() that can write from a
socket to a diskfile also.

-Dan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: / on ramfs, possible?

[Stuart Lynne]

In article <200010300727.IAA12250@hell.wii.ericsson.net>,
Anders Eriksson <aer-list@mailandnews.com> wrote:
>--==_Exmh_17293564P
>Content-Type: text/plain; charset=us-ascii
>
>
>I want my / to be a ramfs filesystem. I intend to populate it from an 
>initrd image, and then remount / as the ramfs filesystem. Is that at 
>all possible? The way I see it the kernel requires / on a device 
>(major,minor) or nfs.
>
>Am I out of luck using ramfs as /? If it's easy to fix, how do I fix it?

Yes it works.

You will need pivot_root. 

Something like the following at the end of your initrd /linuxrc script 
should mount your ramfs, copy the existing root fs files to it, pivot
and unmount your old root. YMMV
 
    mkdir -p /ramfs /ram1
    mount -t ramfs /ramfs /ramfs
    find / | sed '/^\/ramfs/d;/^\/proc\/.*/d' | cpio -pdmV /ramfs
    cd /ramfs
    pivot_root . ram1
    exec chroot . sh -c 'umount /ram1; exit' < dev/console >dev/console


BTW has anyone thought of writing a small utility to emulate df for ramfs?

-- 
                                            __O 
Fireplug - a Lineo company                _-\<,_ 
PGP Fingerprint: 28 E2 A0 15 99 62 9A 00 (_)/ (_) 88 EC A3 EE 2D 1C 15 68
Stuart Lynne <sl@fireplug.net>       www.fireplug.net        604-461-7532
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Recommended compiler? - Re: [patch] kernel/module.c (plus gratuitous rant)

[Horst von Brand]

Martin Dalecki <dalecki@evision-ventures.com> said:
> Peter Samuelson wrote:

[...]

> > * Red Hat "2.96" or CVS 2.97 will probably break any known kernel.

> Works fine for me and 2.4.0-test10-pre5... however there are tons of
> preprocessor warnings in some drivers.

CVS (from 20001028 or so) gave a 2.4.0.10.6/i686 that crashed on boot, no
time to dig deeper yet.
-- 
Dr. Horst H. von Brand                       mailto:vonbrand@inf.utfsm.cl
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Thanks,

It will make merging the MANOS kernel happen faster.  My DLL prototypes
are using subsets
of Linux 2.2.16 for MANOS at present, and what I really need is for the
support issues to dovetail into a supported effort.  This one might fit
the bill.  I have no desire for TRG to support the 100's of LAN and disk
drivers all by our little lonesome in a divergent code base.

Jeff

Alan Cox wrote:
> 
> > context switches.   profiling Ring 0 Linux vs. NetWare will give me an
> > excellent idea of where
> > the optimizations will need to be inserted.  A straight MARS-NWE port to
> > kernel would just
> > happen, since we would be able to just load in kernel space and run it
> > with no code
> > changes.
> 
> There are one bunch of people running Linux on a flat memory space with no
> protection although their goal was to make Linux run on mmuless embedded
> hardware.
> 
> See www.uclinux.org; the uclinux guys started a 2.4 port recently. Basically
> the idea is to have a mm-nommu/ directory which implements a mostly compatible
> replacement for the mm layer (obviously stuff like mmap dont work without an
> mmu and fork is odd), and a set of binary loaders to load flat binaries with
> relocations.
> 
> That I think is the project that overlaps ..
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Recommended compiler? - Re: [patch] kernel/module.c (plus gratuitous rant)

[Jakub Jelinek]

On Mon, Oct 30, 2000 at 05:50:07PM -0300, Horst von Brand wrote:
> Martin Dalecki <dalecki@evision-ventures.com> said:
> > Peter Samuelson wrote:
> 
> [...]
> 
> > > * Red Hat "2.96" or CVS 2.97 will probably break any known kernel.
> 
> > Works fine for me and 2.4.0-test10-pre5... however there are tons of
> > preprocessor warnings in some drivers.
> 
> CVS (from 20001028 or so) gave a 2.4.0.10.6/i686 that crashed on boot, no
> time to dig deeper yet.

CVS 2.97 is known to miscompile e.g. buffer.c.

	Jakub
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

[PATCH] ipv4 skbuff locking scope

[Tom Leete]

[-- Attachment #1: Type: text/plain, Size: 174 bytes --]

Hello,

This fixes tests of a socket buffer done without holding the
lock. tcp_data_wait() and wait_for_tcp_memory() both had
unguarded refs in their sleep conditionals.

Tom

[-- Attachment #2: tcp.lock.scope.patch --]
[-- Type: text/plain, Size: 1387 bytes --]

--- linux-2.4.0-test10-pre5/net/ipv4/tcp.c~	Sun Oct 29 01:21:09 2000
+++ linux/net/ipv4/tcp.c	Mon Oct 30 16:53:19 2000
@@ -204,6 +204,9 @@
  *		Andi Kleen 	:	Make poll agree with SIGIO
  *	Salvatore Sanfilippo	:	Support SO_LINGER with linger == 1 and
  *					lingertime == 0 (RFC 793 ABORT Call)
+ *              Tom Leete       :       Fix locking scope in
+ *                                      wait_for_tcp_memory, tcp_data_wait
+ *
  *					
  *		This program is free software; you can redistribute it and/or
  *		modify it under the terms of the GNU General Public License
@@ -871,10 +874,11 @@
 			break;
 		if (sk->err)
 			break;
-		release_sock(sk);
-		if (!tcp_memory_free(sk) || vm_wait)
+		if (!tcp_memory_free(sk) || vm_wait) {
+			release_sock(sk);
 			current_timeo = schedule_timeout(current_timeo);
-		lock_sock(sk);
+			lock_sock(sk);
+		}
 		if (vm_wait) {
 			if (timeo != MAX_SCHEDULE_TIMEOUT &&
 			    (timeo -= vm_wait-current_timeo) < 0)
@@ -1273,12 +1277,12 @@
 	__set_current_state(TASK_INTERRUPTIBLE);
 
 	set_bit(SOCK_ASYNC_WAITDATA, &sk->socket->flags);
-	release_sock(sk);
 
-	if (skb_queue_empty(&sk->receive_queue))
+	if (skb_queue_empty(&sk->receive_queue)){
+		release_sock(sk);
 		timeo = schedule_timeout(timeo);
-
-	lock_sock(sk);
+		lock_sock(sk);
+	}
 	clear_bit(SOCK_ASYNC_WAITDATA, &sk->socket->flags);
 
 	remove_wait_queue(sk->sleep, &wait);

Re: [PATCH] ipv4 skbuff locking scope

[David S. Miller]

   Date: Mon, 30 Oct 2000 17:24:24 -0500
   From: Tom Leete <tleete@mountain.net>

   This fixes tests of a socket buffer done without holding the
   lock. tcp_data_wait() and wait_for_tcp_memory() both had
   unguarded refs in their sleep conditionals.

These are not buggy at all, see the discussion which took place here
over the past few days.

Look, if the sleep condition test "races" due to not holding the lock,
the schedule() just returns because if the sleep condidion test passes
then by definition this means we were also woken up, see?

BTW, while we're on the topic of people not understanding the
networking locking and proposing bogus patches, does anyone know who
sent the bogon IP tunneling locking "fixes" to Linus behind my back?

They were crap too, and I had to revert them in test10-pre7.  It's
another case of people just not understanding how the code works and
thus that it is correct without any changes.

Please send such fixes to me, and I'll set you straight with a
description as to why your change is unnecessary :-)

Later,
David S. Miller
davem@redhat.com

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: / on ramfs, possible?

[David Woodhouse]

On 29 Oct 2000, H. Peter Anvin wrote:

> > I want my / to be a ramfs filesystem. I intend to populate it from an 
> > initrd image, and then remount / as the ramfs filesystem. Is that at 
> > all possible? The way I see it the kernel requires / on a device 
> > (major,minor) or nfs.
> > 
> > Am I out of luck using ramfs as /? If it's easy to fix, how do I fix it?
> > 
> 
> Use pivot_root instead of the initrd stuff in /proc/sys.

Urgh. Then you're still using an initrd, and you still have to include all
the crap necessary to support those horrid block-device thingies. 

Why not just use a ramdisk?

-- 
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[David Woodhouse]

On Mon, 30 Oct 2000, Ingo Molnar wrote:

> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> > Is there an option to map Linux into a flat address space [...]
> 
> nope, Linux is fundamentally multitasked.

uClinux may be able to do this, at the cost of a dramatically reduced
userspace functionality.

-- 
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: / on ramfs, possible?

[H. Peter Anvin]

David Woodhouse wrote:
> 
> On 29 Oct 2000, H. Peter Anvin wrote:
> 
> > > I want my / to be a ramfs filesystem. I intend to populate it from an
> > > initrd image, and then remount / as the ramfs filesystem. Is that at
> > > all possible? The way I see it the kernel requires / on a device
> > > (major,minor) or nfs.
> > >
> > > Am I out of luck using ramfs as /? If it's easy to fix, how do I fix it?
> > >
> >
> > Use pivot_root instead of the initrd stuff in /proc/sys.
> 
> Urgh. Then you're still using an initrd, and you still have to include all
> the crap necessary to support those horrid block-device thingies.
> 
> Why not just use a ramdisk?
> 

Pardon?!  This doesn't make any sense...

The question was: how do switch from the initrd to using the ramfs as /? 
Using pivot_root should do it (after the pivot, you can of course nuke
the initrd ramdisk.)

	-hpa

-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: / on ramfs, possible?

[David Woodhouse]

On Mon, 30 Oct 2000, H. Peter Anvin wrote:

> Pardon?!  This doesn't make any sense...
> 
> The question was: how do switch from the initrd to using the ramfs as /? 
> Using pivot_root should do it (after the pivot, you can of course nuke
> the initrd ramdisk.)

My question is: What do you want to do that for? You can nuke the initrd
ramdisk, but you can't drop the rd.c code, or ll_rw_blk.c code, etc. So
why not just keep your root filesystem in the initrd where it started off?

-- 
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: / on ramfs, possible?

[H. Peter Anvin]

David Woodhouse wrote:
> 
> On Mon, 30 Oct 2000, H. Peter Anvin wrote:
> 
> > Pardon?!  This doesn't make any sense...
> >
> > The question was: how do switch from the initrd to using the ramfs as /?
> > Using pivot_root should do it (after the pivot, you can of course nuke
> > the initrd ramdisk.)
> 
> My question is: What do you want to do that for? You can nuke the initrd
> ramdisk, but you can't drop the rd.c code, or ll_rw_blk.c code, etc. So
> why not just keep your root filesystem in the initrd where it started off?
> 

Umm... because the size of a ramdisk is fixed, but the size of a ramfs is
flexible?

I can certainly understand this problem... I might in fact do exactly
this in the next version of my SuperRescue disk.  There, the ramdisk
which is the real root is populated from a .tar.gz file; the initrd is
just there to unpack the .tar.gz file onto the "real" ramdisk; the initrd
is then jettisoned.

Why not just have the real root be the initrd, you ask?  It's too large:
since an initrd needs to exist in both compressed form and uncompressed
form in memory at the same time; it would mean SuperRescue would no
longer work on systems with 64 MB RAM.  If I went to ramfs it might
actually work on systems with 48 MB RAM, albeit you better not need to
much space in / (or conversely, it would suddenly let you put a whole lot
more stuff in /tmp if you have 512 MB.)

	-hpa

-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: / on ramfs, possible?

[Jeff Garzik]

David Woodhouse wrote:
> 
> On Mon, 30 Oct 2000, H. Peter Anvin wrote:
> 
> > Pardon?!  This doesn't make any sense...
> >
> > The question was: how do switch from the initrd to using the ramfs as /?
> > Using pivot_root should do it (after the pivot, you can of course nuke
> > the initrd ramdisk.)
> 
> My question is: What do you want to do that for? You can nuke the initrd
> ramdisk, but you can't drop the rd.c code, or ll_rw_blk.c code, etc. So
> why not just keep your root filesystem in the initrd where it started off?

ramfs size is far more dynamic than rd, and it shrinks as well as grows.

Unless you are creating a lot of temporary files and such, though,
initrd is indeed a much better solution from many perspectives. (IMHO)

	Jeff


-- 
Jeff Garzik             | "Mind if I drive?"  -Sam
Building 1024           | "Not if you don't mind me clawing at the
MandrakeSoft            |  dash and shrieking like a cheerleader."
                        |                     -Max
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

David/Alan,

Andre Hedrick is now the CTO of TRG and Chief Scientist over Linux
Development.  After talking 
to him, we are going to do our own ring 0 2.4 and 2.2.x code bases for
the MANOS merge.  
the uClinux is interesting, but I agree is limited.  

MANOS schedules should be unaffected.  The current DLL prototype of
Linux 2.2 is ring 0, but I shudder at trying to merge all the changes
I've done to it into core 2.2.X as a .config
option.  There's also the gravity well forces of different views to this
effort.  With Andre 
on the job, I am more confident in co-opting the Linux drivers and just
biting the bullet 
on the support issues, and doing a full fork of Linux.

Jeff

David Woodhouse wrote:
> 
> On Mon, 30 Oct 2000, Ingo Molnar wrote:
> 
> > On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> >
> > > Is there an option to map Linux into a flat address space [...]
> >
> > nope, Linux is fundamentally multitasked.
> 
> uClinux may be able to do this, at the cost of a dramatically reduced
> userspace functionality.
> 
> --
> dwmw2
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [PATCH] ipv4 skbuff locking scope

[Horst von Brand]

"David S. Miller" <davem@redhat.com> said:

[...]

> Please send such fixes to me, and I'll set you straight with a
> description as to why your change is unnecessary :-)

Could you please Cc: them into the kernel? ;-)
-- 
Horst von Brand                             vonbrand@sleipnir.valparaiso.cl
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [PATCH] ipv4 skbuff locking scope

[Tom Leete]

"David S. Miller" wrote:
> 
>    Date: Mon, 30 Oct 2000 17:24:24 -0500
>    From: Tom Leete <tleete@mountain.net>
> 
>    This fixes tests of a socket buffer done without holding the
>    lock. tcp_data_wait() and wait_for_tcp_memory() both had
>    unguarded refs in their sleep conditionals.
> 
> These are not buggy at all, see the discussion which took place here
> over the past few days.

I'll post traces privately. It was my lockups that got Rick
interested.
 
> Look, if the sleep condition test "races" due to not holding the lock,
> the schedule() just returns because if the sleep condidion test passes
> then by definition this means we were also woken up, see?

Would you explain what is accomplished by toggling the lock
every time through? What breaks by not doing so?

> BTW, while we're on the topic of people not understanding the
> networking locking and proposing bogus patches, does anyone know who
> sent the bogon IP tunneling locking "fixes" to Linus behind my back?

Not me, but see below.
 
> They were crap too, and I had to revert them in test10-pre7.  It's
> another case of people just not understanding how the code works and
> thus that it is correct without any changes.

If it's perfect why can't I use it without locking up the
machine? BTW, I'm currently running my patch. I can now
flood ping 100000 packets in either direction. With
unmodified tcp that is a red switcher (reliably hard locks
all i/o including the serial console).

> Please send such fixes to me, and I'll set you straight with a
> description as to why your change is unnecessary :-)

Perhaps that's why somebody wants to bypass you.

Tom Leete
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [PATCH] ipv4 skbuff locking scope

[David S. Miller]

   Date: Mon, 30 Oct 2000 22:18:07 -0500
   From: Tom Leete <tleete@mountain.net>

   > Look, if the sleep condition test "races" due to not holding the
   > lock, the schedule() just returns because if the sleep condidion
   > test passes then by definition this means we were also woken up,
   > see?

   Would you explain what is accomplished by toggling the lock every
   time through? What breaks by not doing so?

Incoming packets are not processed while the lock is held, they
get queued to the backlog.  When the lock is released, the backlog
packets get processed.

For a TCP sender, these backlog packets are ACKs making room in the
send buffer so that the application can put more data into the send
buffer.

Later,
David S. Miller
davem@redhat.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [PATCH] ipv4 skbuff locking scope

[Albert D. Cahalan]

David S. Miller writes:

> BTW, while we're on the topic of people not understanding the
> networking locking and proposing bogus patches, does anyone know who
> sent the bogon IP tunneling locking "fixes" to Linus behind my back?
...
> Please send such fixes to me, and I'll set you straight with a
> description as to why your change is unnecessary :-)

You can prevent future "fixes" by putting your comments in the code.

/*  like this  */
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Mon, 30 Oct 2000, Jeff V. Merkey wrote:

> Ingo's helping me get the info together on this for putting a MARS-NWE
> tux module in the kernel. [...]

TUX modules are user-space, so i certainly cannot help you in 'putting
MARS-NWE in the kernel'. While you (apparently) are trying to move server
applications into ring0, i agree with Andrea and i'm trying to move kernel
functionality out to user-space.

> He had to go do some things this week he told me before he would be
> ready to look at it. He did point me over to the info, and I agreed we
> would attempt to implement it as something to look at. If it performs
> well enough, I will have something reasonable to send out to Novell
> Resellers (CNEs) and Cutomers.

All i did was to inform you that the next release of TUX is imminent and
that you might want to take a look at the new code. You interpreted that
in a very interesting way. You are certainly free and welcome to take a
look at any code and documentation released, but as visible in the past
couple of email exchanges, our technical views about Linux networking
scalability differ in fundamental ways.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Erik Andersen]

On Mon Oct 30, 2000 at 06:34:55PM +0000, Alan Cox wrote:
> 
> See www.uclinux.org; the uclinux guys started a 2.4 port recently. Basically
> the idea is to have a mm-nommu/ directory which implements a mostly compatible
> replacement for the mm layer (obviously stuff like mmap dont work without an
> mmu and fork is odd), and a set of binary loaders to load flat binaries with
> relocations.

mmap works -- you just can't do MAP_PRIVATE.  Everything has to be
MAP_SHARED.  There is no fork (though vfork works).  brk doesn't work,
and things like iopl, ioperm are pointless.  etc...

Oh, and when you do malloc something (malloc is mmap based) you
really want to remember to free it, since the system can't clean
up after you,

 -Erik

--
Erik B. Andersen   email:  andersee@debian.org
--This message was written using 73% post-consumer electrons--
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Recommended compiler? - Re: [patch] kernel/module.c (plus gratuitous rant)

[Martin Dalecki]

Horst von Brand wrote:
> 
> Martin Dalecki <dalecki@evision-ventures.com> said:
> > Peter Samuelson wrote:
> 
> [...]
> 
> > > * Red Hat "2.96" or CVS 2.97 will probably break any known kernel.
> 
> > Works fine for me and 2.4.0-test10-pre5... however there are tons of
> > preprocessor warnings in some drivers.
> 
> CVS (from 20001028 or so) gave a 2.4.0.10.6/i686 that crashed on boot, no
> time to dig deeper yet.

I was just using the compiler shipped by RedHat with all the fixes
contained
therein.... self compiled under glibc-2.1.95 on a system which some long
time
ago was RedHat-5.1 ;-). And it worked.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Quota mods needed for journaled quota

[Jan Kara]

  Hello.

> There are a few problems in the Linux quota code which make it
> impossible to perform quota updates transactionally when using a
> journaled filesystem.
> 
> Basically we have the following problems:
> 
>  * The underlying filesystem does not know which files are the quota
>    files, so cannot tell when to apply journaling consistency
>    guarantees to data writes
> 
>  * "chown" is not transactional: the filesystem is given no
>    opportunity to wrap the quota transfer and the owner-attribute
>    notify-change call into a single transaction.
  In my quota fix this has changed a bit as now filesystem's notify_change()
is calling dquot_transfer() (it was needed due to some races and I think it's
also more consistent with how other quota calls work). So this problem
should exist no more.
  
> All of these could be fixed very easily (at least for ext3) if it were
> possible for ext3 to install its own version of the superblock->dq_ops
> quota operations (which would just be simple wrappers around the
> existing quota calls).  However, the current sys_quotactl installs the
> default quota_ops into the superblock on quota_on without any chance
> for the filesystem to override it.
> 
> The addition of an "init_quota" method to the super_operations struct,
> with quota_on calling this and defaulting to installing the default
> quota_ops if the method is NULL, ought to be sufficient to let ext3
> get quotas right in all cases as far as I can see.
  Actually I was proposing something similar in my reaction on dquot hang in
ext3... The only difference was that I proposed to install dq_ops in
foo_read_super() so that it will be more consistent with how other callbacks
work. And in my fix of quota I removed testing of dq_ops being NULL so
operations can be set during all life (mount time) of filesystem.

								Honza
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Reto Baettig]

When I'm following this thread, you guys seem to forget the _basics_:
The Linux networking stack sucks!

Everybody tries to work around the networking stack. We just recently
developped a rpc protocol which makes 180MBytes/second (over a Quadrics
Network) because the linux network layer was way too slow. At speeds
above 100MBytes/s, copies start to hurt.

Why not solve the problem at the source and completely redesign the
network stack? Get rid of the old sk_buff & co! Rip the whole network
layer out! Redesign it and give the user a possibility of Zero-Copy
networking!

Reto
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Quota mods needed for journaled quota

[Jan Kara]

  Hello.

> On Oct 26, 11:00am, Stephen C. Tweedie wrote:
> > Subject: Re: Quota mods needed for journaled quota
> > ...
> > > This would allow ext3 to do that which it needs to do differently
> > > at Q_QUOTAON and would also allow Jan's changes to work in such
> > > a way that both the current form of dquot structure and his new
> > > version of dquots could be used together
> > 
> > Adding the init_quota hook would do that, as the filesystem will be
> > able to install its own dq_ops methods during the init so we get the
> > flexibility you are asking for anyway.
> > 
> 
> Hmmm ... I'm not so sure.  In order to have the flexibility
> of filesystem-specific dquot formats, the struct dquot would
> need to become more like struct inode/super_block, i.e. not
> hardcoding the ondisk structure into the incore structure
> (using a union and a generic pointer, as inode/super_block do).
> 
> The DQUOT_SYNC mechanism would need to be able to be overridden
> per-filesystem also.  It isn't really as cut-and-dried as "per-
> filesystem" either, because an ext2/3 filesystem might make use
> of either the original dquot format or Jan's newer format, either
> at mount time or even after doing a quota_off & quota_on with a
> new quota file format (that would be quite clean).
  Hmm. Probably I wouldn't allow to override quotactl() but make it like
other callbacks - operations like quota_on() quota_off() and so
could be overridden (or better filesystem could specify callback to be called
after some generic work), quotactl() will call foo_quotactl() if it won't
recognize the operation number.
  But I don't feel urgent need of this redesign so I would wait for some
time so current fixes can settle down...

								Honza

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Reto Baettig]

Alan Cox wrote:
> 
> > Why not solve the problem at the source and completely redesign the
> > network stack? Get rid of the old sk_buff & co! Rip the whole network
> > layer out! Redesign it and give the user a possibility of Zero-Copy
> > networking!
> 
> For one because you don't need to do that to get zero copy networking for
> most real world cases. Tux already implements the neccessary infrastructure
> for these.

And what if I'd like to use the network for something different than
html?

I'm sorry but I don't know TUX. Does it implement its own TCP stack? 

Reto
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Reto Baettig]

Rik van Riel wrote:
> Ummm, last I looked Linux held the Specweb99 record;
> by a wide margin...

...does that remove any memory copies???

To be best does not mean that there's no place for improvment.

Can anybody please help me and tell me where to start understanding what
tux does?

www.tux.org does not seem to be the right place :-(

I don't want to make linux bad or stand on anybodys toes. I'd just like
to improve linux like everybody else and I do not know every single
peace of source code floating around the world by heart ;-)

Reto
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Reto Baettig]

Rik

Is there any documentation about the Tux zero-copy implementation so
that I don't have to read half of the 2.4 kernel sources before having a
clue? 

Are the kernel changes going to be in the mainstream kernel?

Does Tux implement a new interface so that a userspace app can do
zero-copy stuff with the network?

Sorry for the newbie questions, but it's really hard to find information
about Tux (other than the holy source code, of course ;-)

TIA

Reto

Rik van Riel wrote:
> 
> On Tue, 31 Oct 2000, Jeff V. Merkey wrote:
> > Rik van Riel wrote:
> > > On Tue, 31 Oct 2000, Reto Baettig wrote:
> > >
> > > > When I'm following this thread, you guys seem to forget the
> > > > _basics_: The Linux networking stack sucks!
> > >
> > > Ummm, last I looked Linux held the Specweb99 record;
> > > by a wide margin...
> >
> > It doesn't hold the file and print scaling record.  NetWare
> > does..
> 
> Indeed, we haven't made a file serving plugin for
> the TUX zero-copy stuff yet...
> 
> Oh, and I haven't found a bunch of printers yet that are
> fast enough to beat the printserving record ;))
>   *runs like hell*
> 
> cheers,
> 
> Rik
> --
> "What you're running that piece of shit Gnome?!?!"
>        -- Miguel de Icaza, UKUUG 2000
> 
> http://www.conectiva.com/               http://www.surriel.com/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Pavel Machek]

Hi!

> > > This is putrid. NetWare does 353,00,000/second on a Xenon, pumping out
> > > gobs of packets in between them. MANOS does 857,000,000/second. This
> > > is terrible. No wonder it's so f_cking slow!!!
> 
> And please check your numbers, 857 million
> > context switches per second means that on a 1 GHZ CPU you do one context
> > switch per 1.16 clock cycles. Wow!
> 
> Excuse me, 857,000,000 instructions executed and 460,000,000 context
> switches
> a second -- on a PII system at 350 Mhz.  It's due to AGI
> optimization.  

That's more than one context switch per clock. I do not think
so. Really go and check those numbers.
								Pavel
-- 
I'm pavel@ucw.cz. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at discuss@linmodems.org
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Pavel Machek]

Hi!

> > TUX modules are kernel modules (I mean you have to write kernel space code for
> > doing TUX ftp). Don't you agree that zero-copy sendfile like ftp serving would
> > be able to perform equally well too?
> 
> For this to bw useful for ftp we need a sendfile() that can write from a
> socket to a diskfile also.

I had patch to fix sendfile this way... Sendfile is really ugly, as of
now. (It basically falled back to read/write, giving only small
performance advantage, but it made things cleaner).

								Pavel
-- 
I'm pavel@ucw.cz. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at discuss@linmodems.org
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Ingo Molnar wrote:
> 
> On Mon, 30 Oct 2000, Jeff V. Merkey wrote:
> 
> 
> All i did was to inform you that the next release of TUX is imminent and
> that you might want to take a look at the new code. You interpreted that
> in a very interesting way. 

I seem to remember a "don't post this email on the list here's what's
up" message
from you I that I honored your instructions and did not post, nor will
I, but I recall 
some reasonable suggestions from it. :-)

You are certainly free and welcome to take a
> look at any code and documentation released, but as visible in the past
> couple of email exchanges, our technical views about Linux networking
> scalability differ in fundamental ways.

I've been working on LAN scalability for almost 20 years, and NetWare
LAN performance vs. Linux (or TUX) speaks for itself.  I would agree
that someone 
coming from a Unix background would have radically different views than
someone 
coming from the group who created the Networking industry that made
"LAN" a household 
word to the world at large.

It's the differences of all those who work on Linux that make it such a
potent 
mix of skills and views.  I think this is a very good thing, and I take
your comment 
as a compliment.

:-)

Jeff

> 





>         Ingo
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Pavel Machek wrote:
> 
> Hi!
> 
> > > > This is putrid. NetWare does 353,00,000/second on a Xenon, pumping out
> > > > gobs of packets in between them. MANOS does 857,000,000/second. This
> > > > is terrible. No wonder it's so f_cking slow!!!
> >
> > And please check your numbers, 857 million
> > > context switches per second means that on a 1 GHZ CPU you do one context
> > > switch per 1.16 clock cycles. Wow!
> >
> > Excuse me, 857,000,000 instructions executed and 460,000,000 context
> > switches
> > a second -- on a PII system at 350 Mhz.  It's due to AGI
> > optimization.
> 
> That's more than one context switch per clock. I do not think
> so. Really go and check those numbers.


Pavel,  The optimization exploits the multiple piplines in Intel's
processors,
and yes, it does execute more than one instruction per clock, it's
optimized
to execute in the processors parallel pipelines.  The EMON numbers are
accurate,
and you can download the kernel and verify for yourself.  These types of
optimizations
are possible when people have acccess to Intel Red Cover documents, then
you 
get to know just how Intel's internal architectures are affected by 
different coding optimizations.

Jeff  
>                                                                 Pavel
> --
> I'm pavel@ucw.cz. "In my country we have almost anarchy and I don't care."
> Panos Katsaloulis describing me w.r.t. patents at discuss@linmodems.org
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

"Jeff V. Merkey" wrote:
> 
> Pavel Machek wrote:
> >
> > Hi!
> >
> > > > > This is putrid. NetWare does 353,00,000/second on a Xenon, pumping out
> > > > > gobs of packets in between them. MANOS does 857,000,000/second. This
> > > > > is terrible. No wonder it's so f_cking slow!!!
> > >
> > > And please check your numbers, 857 million
> > > > context switches per second means that on a 1 GHZ CPU you do one context
> > > > switch per 1.16 clock cycles. Wow!
> > >
> > > Excuse me, 857,000,000 instructions executed and 460,000,000 context
> > > switches
> > > a second -- on a PII system at 350 Mhz.  It's due to AGI
> > > optimization.
> >
> > That's more than one context switch per clock. I do not think
> > so. Really go and check those numbers.
> 
> Pavel,  The optimization exploits the multiple piplines in Intel's
> processors,
> and yes, it does execute more than one instruction per clock, it's
> optimized
> to execute in the processors parallel pipelines.  The EMON numbers are
> accurate,
> and you can download the kernel and verify for yourself.  These types of
> optimizations
> are possible when people have acccess to Intel Red Cover documents, then
> you
> get to know just how Intel's internal architectures are affected by
> different coding optimizations.
> 
> Jeff

There's also another optimization in this kernel that allows it to
achieve greater than
100% scaling per processor by using a strong affinity algorithm (I hold
the patent 
on this algorithimn, and by posting code based on it under the GPL, I
have released
it to the general public).  

It relies on an anomoly in the design of Intel's cache controllers, and
with memory based
applications, I can get 120% scaling per procesoor by jugling the
working set of 
executable code cached accros each processor.  There's sample code with
this kernel 
you can use to verify....

:-)

Jeff


> >                                                                 Pavel
> > --
> > I'm pavel@ucw.cz. "In my country we have almost anarchy and I don't care."
> > Panos Katsaloulis describing me w.r.t. patents at discuss@linmodems.org
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Alan Cox]

> Why not solve the problem at the source and completely redesign the
> network stack? Get rid of the old sk_buff & co! Rip the whole network
> layer out! Redesign it and give the user a possibility of Zero-Copy
> networking!

For one because you don't need to do that to get zero copy networking for
most real world cases. Tux already implements the neccessary infrastructure
for these.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Rik van Riel]

On Tue, 31 Oct 2000, Reto Baettig wrote:

> When I'm following this thread, you guys seem to forget the
> _basics_: The Linux networking stack sucks!

Ummm, last I looked Linux held the Specweb99 record;
by a wide margin...

Rik
--
"What you're running that piece of shit Gnome?!?!"
       -- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/		http://www.surriel.com/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Alan Cox]

> And what if I'd like to use the network for something different than
> html?

Read the tux source. Then come back and ask sensible questions


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jesse Pollard]

---------  Received message begins Here  ---------

> 
> > And what if I'd like to use the network for something different than
> > html?
> 
> Read the tux source. Then come back and ask sensible questions

Also pay attention to the security aspects of a true "zero copy" TCP stack.
It means that SOMETIMES a user buffer will recieve data that is destined
for a different process.

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Alan Cox]

> Also pay attention to the security aspects of a true "zero copy" TCP stack.
> It means that SOMETIMES a user buffer will recieve data that is destined
> for a different process.

The moment you try and do zero copy like that you end up playing so many MMU
games the copy is faster. We do zero copy from the kernel side of the universe
thats a lot lot saner

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Rik van Riel]

On Tue, 31 Oct 2000, Reto Baettig wrote:
> Rik van Riel wrote:
> > Ummm, last I looked Linux held the Specweb99 record;
> > by a wide margin...
> 
> ...does that remove any memory copies???

> I don't want to make linux bad or stand on anybodys toes.

Good to know, your previous message might have fooled some
people when it comes to these intentions ;)

--------------
On Tue, 31 Oct 2000, Reto Baettig wrote:

> When I'm following this thread, you guys seem to forget the
> _basics_: The Linux networking stack sucks!
--------------

cheers,

Rik
--
"What you're running that piece of shit Gnome?!?!"
       -- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/		http://www.surriel.com/


Rik
--
"What you're running that piece of shit Gnome?!?!"
       -- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/		http://www.surriel.com/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Tue, 31 Oct 2000, Jeff V. Merkey wrote:

> It relies on an anomoly in the design of Intel's cache controllers,
> and with memory based applications, I can get 120% scaling per
> procesoor by jugling the working set of executable code cached accros
> each processor.  There's sample code with this kernel you can use to
> verify....

FYI, this is a very old concept and a scalability FAQ item. It's called
"sublinear scaling", and SGI folks have already published articles about
it 10 years ago.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Rik van Riel wrote:
> 
> On Tue, 31 Oct 2000, Reto Baettig wrote:
> 
> > When I'm following this thread, you guys seem to forget the
> > _basics_: The Linux networking stack sucks!
> 
> Ummm, last I looked Linux held the Specweb99 record;
> by a wide margin...
> 

It doesn't hold the file and print scaling record.  NetWare does..

Jeff

> Rik
> --
> "What you're running that piece of shit Gnome?!?!"
>        -- Miguel de Icaza, UKUUG 2000
> 
> http://www.conectiva.com/               http://www.surriel.com/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Tue, 31 Oct 2000, Pavel Machek wrote:

> > Excuse me, 857,000,000 instructions executed and 460,000,000
> > context switches a second -- on a PII system at 350 Mhz. [...]

> That's more than one context switch per clock. I do not think so.
> Really go and check those numbers.

yep, you cannot have 460 million context switches on that system,
unless you have some Clintonesque definition for 'context switch' ;-)

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Alan Cox wrote:
> 
> > Why not solve the problem at the source and completely redesign the
> > network stack? Get rid of the old sk_buff & co! Rip the whole network
> > layer out! Redesign it and give the user a possibility of Zero-Copy
> > networking!
> 
> For one because you don't need to do that to get zero copy networking for
> most real world cases. Tux already implements the neccessary infrastructure
> for these.

The code in the networking layer is fine, in fact, it's absolutely
great.  This is not
the problem.   The problem are all the clocks wasted reloading TLBs and
the background 
memory activitiy caused by Linux's heavy dependence in Intel's
protection hardware model.

Step 1 is to load the entire OS at ring 0 with protection completely
disabled system wide, 
put in the kernel optimizations for AGI and context switch locking, and
stub the top half of Linus's scheduler.  The global memory structures in
his kernel may or may not hurt performance, depending on how they are
accessed by multiple processors.  I will need to start profiling with a
ring 0 port first and determine frequency of access that's hardware
measured.

The next step would be to start peeling off different subsystems and
re-parallelizing them 
on the merged kernel.  There's an awful lot of areas in Linus' that are
going to be a 
problem, but I'll work through them one at a time.  When I first
parallelized NetWare, I wrote
an independent SMP kernel then loaded the entire NetWare 4.1 image as a
single process.  The next step was to start peeling off layers from
NetWare and plugging them in one by one and parallelizing them.  I am
using the same approach here.  When I was finished, I had peeled NetWare
like a banana and completely reintegrated it on a new kernel.  This
approach works because it allows you to stage each layer.

The next step will be to modify the loader to allow the existing
protection scheme to exist in true user space.  WRD's will hold off CR3
switching so long as I/O is coming in or out of the box.  I anticipate
this will take until March of next year to get right.

Jeff






packets via a WTD scheduler,
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Rik van Riel]

On Tue, 31 Oct 2000, Jeff V. Merkey wrote:
> Rik van Riel wrote:
> > On Tue, 31 Oct 2000, Reto Baettig wrote:
> > 
> > > When I'm following this thread, you guys seem to forget the
> > > _basics_: The Linux networking stack sucks!
> > 
> > Ummm, last I looked Linux held the Specweb99 record;
> > by a wide margin...
> 
> It doesn't hold the file and print scaling record.  NetWare
> does..

Indeed, we haven't made a file serving plugin for
the TUX zero-copy stuff yet...

Oh, and I haven't found a bunch of printers yet that are
fast enough to beat the printserving record ;))
  *runs like hell*

cheers,

Rik
--
"What you're running that piece of shit Gnome?!?!"
       -- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/		http://www.surriel.com/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

"Jeff V. Merkey" wrote:
> 
> Alan Cox wrote:
> >
> > > Why not solve the problem at the source and completely redesign the
> > > network stack? Get rid of the old sk_buff & co! Rip the whole network
> > > layer out! Redesign it and give the user a possibility of Zero-Copy
> > > networking!
> >
> > For one because you don't need to do that to get zero copy networking for
> > most real world cases. Tux already implements the neccessary infrastructure
> > for these.
> 
> The code in the networking layer is fine, in fact, it's absolutely
> great.  This is not
> the problem.   The problem are all the clocks wasted reloading TLBs and
> the background
> memory activitiy caused by Linux's heavy dependence in Intel's
> protection hardware model.
> 
> Step 1 is to load the entire OS at ring 0 with protection completely
> disabled system wide,
> put in the kernel optimizations for AGI and context switch locking, and
> stub the top half of Linus's scheduler.  The global memory structures in
> his kernel may or may not hurt performance, depending on how they are
> accessed by multiple processors.  I will need to start profiling with a
> ring 0 port first and determine frequency of access that's hardware
> measured.
> 
> The next step would be to start peeling off different subsystems and
> re-parallelizing them
> on the merged kernel.  There's an awful lot of areas in Linus' that are
> going to be a
> problem, but I'll work through them one at a time.  When I first
> parallelized NetWare, I wrote
> an independent SMP kernel then loaded the entire NetWare 4.1 image as a
> single process.  The next step was to start peeling off layers from
> NetWare and plugging them in one by one and parallelizing them.  I am
> using the same approach here.  When I was finished, I had peeled NetWare
> like a banana and completely reintegrated it on a new kernel.  This
> approach works because it allows you to stage each layer.
> 
> The next step will be to modify the loader to allow the existing
> protection scheme to exist in true user space.  WRD's will hold off CR3
> switching so long as I/O is coming in or out of the box.  I anticipate
> this will take until March of next year to get right.
> 
> Jeff
> 


One other point.  You guys can try as hard as you can to hack around
this problem, but it will never scale like NetWare unless a
fundamentally different approach is adopted.  I've built this before,
and what we are doing is building in all the Linux functionality with
the identical code -- basically a Linux NetWare -- into a NetWare
framework.  All the protection can still be there, it just needs to be
restructured with some LAN based design assumptions.  I think the
industry will be pleased with the result.  I do not believe any of the
proposed hacks will get there unless this different approach is
investigated.   

Jeff




> packets via a WTD scheduler,
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Ingo Molnar wrote:
> 
> On Tue, 31 Oct 2000, Pavel Machek wrote:
> 
> > > Excuse me, 857,000,000 instructions executed and 460,000,000
> > > context switches a second -- on a PII system at 350 Mhz. [...]
> 
> > That's more than one context switch per clock. I do not think so.
> > Really go and check those numbers.
> 
> yep, you cannot have 460 million context switches on that system,
> unless you have some Clintonesque definition for 'context switch' ;-)

The numbers don't lie.  You know where the code is.  You notice that
there is a version of
the kernel hand coded in assembly language.  You'l also noticed that
it's SMP and takes ZERO LOCKS during context switching, in fact, most of
the design is completely lockless.

Jeff

> 
>         Ingo
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Rik van Riel wrote:
> 
> On Tue, 31 Oct 2000, Jeff V. Merkey wrote:
> > Rik van Riel wrote:
> > > On Tue, 31 Oct 2000, Reto Baettig wrote:
> > >
> > > > When I'm following this thread, you guys seem to forget the
> > > > _basics_: The Linux networking stack sucks!
> > >
> > > Ummm, last I looked Linux held the Specweb99 record;
> > > by a wide margin...
> >
> > It doesn't hold the file and print scaling record.  NetWare
> > does..
> 
> Indeed, we haven't made a file serving plugin for
> the TUX zero-copy stuff yet...
> 
> Oh, and I haven't found a bunch of printers yet that are
> fast enough to beat the printserving record ;))

You got me on this one.  I would also agree that NPDS is the worst piece
of crap ever written.  

:-)

Jeff


>   *runs like hell*
> 
> cheers,
> 
> Rik
> --
> "What you're running that piece of shit Gnome?!?!"
>        -- Miguel de Icaza, UKUUG 2000
> 
> http://www.conectiva.com/               http://www.surriel.com/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

> "sublinear scaling",
    ^-- extralinear. whatever.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Ingo Molnar wrote:
> 
> On Tue, 31 Oct 2000, Jeff V. Merkey wrote:
> 
> > It relies on an anomoly in the design of Intel's cache controllers,
> > and with memory based applications, I can get 120% scaling per
> > procesoor by jugling the working set of executable code cached accros
> > each processor.  There's sample code with this kernel you can use to
> > verify....
> 
> FYI, this is a very old concept and a scalability FAQ item. It's called
> "sublinear scaling", and SGI folks have already published articles about
> it 10 years ago.

Ingo,

You don't even know what it is enough to comment intelligently.  You can
write the patent office and obtain a copy.  The patent is currently in
dispute between Novell and several other companies over S&E ownership,
and there's a court hearing scheduled to resolve it (lukily I don't have
to deal with this one).  Nice thing about being an inventor, though, is
I have rights to it, no matter who ends up with the S&E assignment.
(dogs fights over a bone ...).

Jeff

> 
>         Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Rik van Riel]

On Tue, 31 Oct 2000, Reto Baettig wrote:

> Is there any documentation about the Tux zero-copy
> implementation so that I don't have to read half of the 2.4
> kernel sources before having a clue?

Reading the 2.4 sources won't do you much good since
the Tux layer isn't integrated ;)

> Are the kernel changes going to be in the mainstream kernel?

Dunno, ask Ingo and/or Linus...

> Sorry for the newbie questions, but it's really hard to find
> information about Tux (other than the holy source code, of
> course ;-)

Maybe there's some documentation included with the Tux
source code or maybe there's something on the Tux web
page?  Have you tried looking there?

regards,

Rik
--
"What you're running that piece of shit Gnome?!?!"
       -- Miguel de Icaza, UKUUG 2000

http://www.conectiva.com/		http://www.surriel.com/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Andi Kleen]

On Tue, Oct 31, 2000 at 02:52:11PM -0700, Jeff V. Merkey wrote:
> The numbers don't lie.  You know where the code is.  You notice that
> there is a version of
> the kernel hand coded in assembly language.  You'l also noticed that
> it's SMP and takes ZERO LOCKS during context switching, in fact, most of
> the design is completely lockless.

I suspect most of the confusion in this thread comes because you seem to 
use a different definition of context switch than Ingo and others. Could
you explain what you exactly mean with a context switch ? 

-Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

A "context" is usually assued to be a "stack".  The simplest of all
context switches 
is:

   mov    x, esp
   mov    esp, y

A context switch can be as short as two instructions, or as big as a TSS
with CR3 hardware switching,

i.e.  

   ltr    ax
   jmp    task_gate

(500 clocks later)

   ts->eip gets exec'd

you can also have a context switch that does an int X where X is a gate
or TSS.

you can also have a context switch (like linux) that does

    mov    x, esp
    mov    esp, y
    mov    z, CR3
    mov    CR3, w

etc.

In NetWare, a context switch is an in-line assembly language macro that
is 2 instructions long for a stack switch and 4 instructions for a CR3
reload -- this is a lot shorter than Linux.
Only EBX, EBP, ESI, and EDI are saved and this is never done in the
kernel, but is a natural
affect of the Watcom C compiler.  There's also strict rules about
register assignments that re enforced between assembler modules in
NetWare to reduce the overhead of a context switch.  The code path is
very complex in NetWare, and priorities and all this stuff exists, but
these code paths are segragated so these types of checks only happen
once in a while and check a pre-calc'd "scoreboard" that is read only
across processors and updated and recal'd by a timer every 18 ticks.

Jeff

   



Andi Kleen wrote:
> 
> On Tue, Oct 31, 2000 at 02:52:11PM -0700, Jeff V. Merkey wrote:
> > The numbers don't lie.  You know where the code is.  You notice that
> > there is a version of
> > the kernel hand coded in assembly language.  You'l also noticed that
> > it's SMP and takes ZERO LOCKS during context switching, in fact, most of
> > the design is completely lockless.
> 
> I suspect most of the confusion in this thread comes because you seem to
> use a different definition of context switch than Ingo and others. Could
> you explain what you exactly mean with a context switch ?
> 
> -Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Ingo Molnar wrote:
> 
> On Tue, 31 Oct 2000, Jeff V. Merkey wrote:
> 
> > > > > Excuse me, 857,000,000 instructions executed and 460,000,000
> > > > > context switches a second -- on a PII system at 350 Mhz. [...]
> > >
> > > > That's more than one context switch per clock. I do not think so.
> > > > Really go and check those numbers.
> > >
> > > yep, you cannot have 460 million context switches on that system,
> > > unless you have some Clintonesque definition for 'context switch' ;-)
> >
> > The numbers don't lie. [...]
> 
> sure ;) I can do infinite context switches! You dont believe? See:
> 
>         #define schedule() do { } while (0)

Actually, I think the compiler would optimize this statement completely
out of the code.

Jeff

> 
> [there is a small restriction, should only be used in single-task
> systems.]
> 
>         Ingo
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[David Lang]

-----BEGIN PGP SIGNED MESSAGE-----

Jeff, one other thing. Linux is not x86 hand-crafted assembler, it's
capable of running on many platforms. are you planning on giving up this 
capability or hand crafting the kernel for each chip?

Linux on x86 is nice (and I do use it a lot) but one of the things that
makes it so useful is that when you do outgrow what youcan do on a x86
platform you cna move to a more powerful platform without having to change
to a different OS.

 David Lang

On Tue, 31 Oct 2000,
Jeff V. Merkey wrote:

> Date: Tue, 31 Oct 2000 15:45:45 -0700
> From: Jeff V. Merkey <jmerkey@timpanogas.org>
> To: Andi Kleen <ak@suse.de>, mingo@elte.hu, Pavel Machek <pavel@suse.cz>,
>      linux-kernel@vger.kernel.org
> Subject: Re: 2.2.18Pre Lan Performance Rocks!
> 
> 
> 
> One more optimization it has.  NetWare never "calls" functions in the
> kernel.  There's a template of register assignments in between kernel
> modules that's very strict (esi contains a WTD head, edi has the target
> thread, etc.) and all function calls are jumps in a linear space. 
> layout of all functions are 16 bytes aligned for speed, and all
> arguments in kernel are passed via registers.  It's a level of
> optimization no C compiler does -- all of it was done by hand, and most
> function s in fast paths are layed out in 512 byte chunks to increases
> speed.  Stack memory activity in the NetWare kernel is almost
> non-existent in almost all of the "fast paths"
> 
> Jeff
> 
> "Jeff V. Merkey" wrote:
> > 
> > A "context" is usually assued to be a "stack".  The simplest of all
> > context switches
> > is:
> > 
> >    mov    x, esp
> >    mov    esp, y
> > 
> > A context switch can be as short as two instructions, or as big as a TSS
> > with CR3 hardware switching,
> > 
> > i.e.
> > 
> >    ltr    ax
> >    jmp    task_gate
> > 
> > (500 clocks later)
> > 
> >    ts->eip gets exec'd
> > 
> > you can also have a context switch that does an int X where X is a gate
> > or TSS.
> > 
> > you can also have a context switch (like linux) that does
> > 
> >     mov    x, esp
> >     mov    esp, y
> >     mov    z, CR3
> >     mov    CR3, w
> > 
> > etc.
> > 
> > In NetWare, a context switch is an in-line assembly language macro that
> > is 2 instructions long for a stack switch and 4 instructions for a CR3
> > reload -- this is a lot shorter than Linux.
> > Only EBX, EBP, ESI, and EDI are saved and this is never done in the
> > kernel, but is a natural
> > affect of the Watcom C compiler.  There's also strict rules about
> > register assignments that re enforced between assembler modules in
> > NetWare to reduce the overhead of a context switch.  The code path is
> > very complex in NetWare, and priorities and all this stuff exists, but
> > these code paths are segragated so these types of checks only happen
> > once in a while and check a pre-calc'd "scoreboard" that is read only
> > across processors and updated and recal'd by a timer every 18 ticks.
> > 
> > Jeff
> > 
> > 
> > 
> > Andi Kleen wrote:
> > >
> > > On Tue, Oct 31, 2000 at 02:52:11PM -0700, Jeff V. Merkey wrote:
> > > > The numbers don't lie.  You know where the code is.  You notice that
> > > > there is a version of
> > > > the kernel hand coded in assembly language.  You'l also noticed that
> > > > it's SMP and takes ZERO LOCKS during context switching, in fact, most of
> > > > the design is completely lockless.
> > >
> > > I suspect most of the confusion in this thread comes because you seem to
> > > use a different definition of context switch than Ingo and others. Could
> > > you explain what you exactly mean with a context switch ?
> > >
> > > -Andi
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
> 

-----BEGIN PGP SIGNATURE-----
Version: PGP 6.5.2

iQEVAwUBOf9LYj7msCGEppcbAQFdZQgAocWtMwnNmmLnfSS+cGHZd8V0IFfmoVb7
fDRoNWMmOzU5g5W8aAudQFqGpGqizBR8/AA9ziqHRfKxwoo5/nuHtEMDpfw0nV5e
ghsd7qtzv1kTk0l5zp6bN2qPlGgs7Ke72od10X6pTGDyuUDQK71YNQ9UUcCv8GEO
2PpPOnCHw3atuQ0hetMNcFfIdvvslTB2+pcVzYxWWGhCYIeWreF8w1qf8XDYQil9
Ih22vmu69LP03RwXkFioikVSK8F8m+31DUBA67exN+R4qXy8+U5ZtyPQ+onIeguh
SnADBNjGjWK2mPyLNSVyAwH6EsIaGzk1QJ5hYULzVFi4zVl3pyRi8w==
=91LL
-----END PGP SIGNATURE-----
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

One more optimization it has.  NetWare never "calls" functions in the
kernel.  There's a template of register assignments in between kernel
modules that's very strict (esi contains a WTD head, edi has the target
thread, etc.) and all function calls are jumps in a linear space. 
layout of all functions are 16 bytes aligned for speed, and all
arguments in kernel are passed via registers.  It's a level of
optimization no C compiler does -- all of it was done by hand, and most
function s in fast paths are layed out in 512 byte chunks to increases
speed.  Stack memory activity in the NetWare kernel is almost
non-existent in almost all of the "fast paths"

Jeff

"Jeff V. Merkey" wrote:
> 
> A "context" is usually assued to be a "stack".  The simplest of all
> context switches
> is:
> 
>    mov    x, esp
>    mov    esp, y
> 
> A context switch can be as short as two instructions, or as big as a TSS
> with CR3 hardware switching,
> 
> i.e.
> 
>    ltr    ax
>    jmp    task_gate
> 
> (500 clocks later)
> 
>    ts->eip gets exec'd
> 
> you can also have a context switch that does an int X where X is a gate
> or TSS.
> 
> you can also have a context switch (like linux) that does
> 
>     mov    x, esp
>     mov    esp, y
>     mov    z, CR3
>     mov    CR3, w
> 
> etc.
> 
> In NetWare, a context switch is an in-line assembly language macro that
> is 2 instructions long for a stack switch and 4 instructions for a CR3
> reload -- this is a lot shorter than Linux.
> Only EBX, EBP, ESI, and EDI are saved and this is never done in the
> kernel, but is a natural
> affect of the Watcom C compiler.  There's also strict rules about
> register assignments that re enforced between assembler modules in
> NetWare to reduce the overhead of a context switch.  The code path is
> very complex in NetWare, and priorities and all this stuff exists, but
> these code paths are segragated so these types of checks only happen
> once in a while and check a pre-calc'd "scoreboard" that is read only
> across processors and updated and recal'd by a timer every 18 ticks.
> 
> Jeff
> 
> 
> 
> Andi Kleen wrote:
> >
> > On Tue, Oct 31, 2000 at 02:52:11PM -0700, Jeff V. Merkey wrote:
> > > The numbers don't lie.  You know where the code is.  You notice that
> > > there is a version of
> > > the kernel hand coded in assembly language.  You'l also noticed that
> > > it's SMP and takes ZERO LOCKS during context switching, in fact, most of
> > > the design is completely lockless.
> >
> > I suspect most of the confusion in this thread comes because you seem to
> > use a different definition of context switch than Ingo and others. Could
> > you explain what you exactly mean with a context switch ?
> >
> > -Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Ingo Molnar wrote:
> 
> On Tue, 31 Oct 2000, Jeff V. Merkey wrote:
> 
> > A "context" is usually assued to be a "stack". [...]
> 
> a very clintonesque definition indeed ;-)
> 
> what is relevant is the latency to switch from one process to another one.
> And this is what we call a context switch. It includes scheduling
> decisions and all sorts of other stuff. You are comparing stack &
> caller-saved register switching performance (which is just a small part of
> context switching and has no standing alone) against full Linux context
> switch performance (this is what i quoted), and thus you have won my
> 'Mindcraft benchmark of the day' award :-)

Ingo,

It kicks Linux's but in LAN I/O scaling.  It would be nice for Linux to
have an incarnation that's competitive.  The only reason people are
still buying NetWare is because nothing out there has come along to
replace it.  That is going to change...

Jeff 


> 
>         Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[David Lang]

I don't doubt it. a port of netware that can run linux apps would be very
useful to people who want to run netware, but this is not the same thing
as what it has sounded like you were working on.

David Lang

 On Tue, 31 Oct 2000, Jeff
V. Merkey wrote:

> Date: Tue, 31 Oct 2000 15:57:23 -0700
> From: Jeff V. Merkey <jmerkey@timpanogas.org>
> To: David Lang <david.lang@digitalinsight.com>
> Cc: linux-kernel@vger.kernel.org
> Subject: Re: 2.2.18Pre Lan Performance Rocks!
> 
> 
> 
> David Lang wrote:
> > 
> > -----BEGIN PGP SIGNED MESSAGE-----
> > 
> > Jeff, one other thing. Linux is not x86 hand-crafted assembler, it's
> > capable of running on many platforms. are you planning on giving up this
> > capability or hand crafting the kernel for each chip?
> > 
> > Linux on x86 is nice (and I do use it a lot) but one of the things that
> > makes it so useful is that when you do outgrow what youcan do on a x86
> > platform you cna move to a more powerful platform without having to change
> > to a different OS.
> 
> How about a NetWare like NOS with all the application support of Linux? 
> I think a lot of people will love it.  Novell's largest customers have
> told me they want to see it, and would deploy it.  I do believe there's
> a market for such a beast.  A very lucrative one...
> 
> Jeff
> 
> > 
> >  David Lang
> > 
> > On Tue, 31 Oct 2000,
> > Jeff V. Merkey wrote:
> > 
> > > Date: Tue, 31 Oct 2000 15:45:45 -0700
> > > From: Jeff V. Merkey <jmerkey@timpanogas.org>
> > > To: Andi Kleen <ak@suse.de>, mingo@elte.hu, Pavel Machek <pavel@suse.cz>,
> > >      linux-kernel@vger.kernel.org
> > > Subject: Re: 2.2.18Pre Lan Performance Rocks!
> > >
> > >
> > >
> > > One more optimization it has.  NetWare never "calls" functions in the
> > > kernel.  There's a template of register assignments in between kernel
> > > modules that's very strict (esi contains a WTD head, edi has the target
> > > thread, etc.) and all function calls are jumps in a linear space.
> > > layout of all functions are 16 bytes aligned for speed, and all
> > > arguments in kernel are passed via registers.  It's a level of
> > > optimization no C compiler does -- all of it was done by hand, and most
> > > function s in fast paths are layed out in 512 byte chunks to increases
> > > speed.  Stack memory activity in the NetWare kernel is almost
> > > non-existent in almost all of the "fast paths"
> > >
> > > Jeff
> > >
> > > "Jeff V. Merkey" wrote:
> > > >
> > > > A "context" is usually assued to be a "stack".  The simplest of all
> > > > context switches
> > > > is:
> > > >
> > > >    mov    x, esp
> > > >    mov    esp, y
> > > >
> > > > A context switch can be as short as two instructions, or as big as a TSS
> > > > with CR3 hardware switching,
> > > >
> > > > i.e.
> > > >
> > > >    ltr    ax
> > > >    jmp    task_gate
> > > >
> > > > (500 clocks later)
> > > >
> > > >    ts->eip gets exec'd
> > > >
> > > > you can also have a context switch that does an int X where X is a gate
> > > > or TSS.
> > > >
> > > > you can also have a context switch (like linux) that does
> > > >
> > > >     mov    x, esp
> > > >     mov    esp, y
> > > >     mov    z, CR3
> > > >     mov    CR3, w
> > > >
> > > > etc.
> > > >
> > > > In NetWare, a context switch is an in-line assembly language macro that
> > > > is 2 instructions long for a stack switch and 4 instructions for a CR3
> > > > reload -- this is a lot shorter than Linux.
> > > > Only EBX, EBP, ESI, and EDI are saved and this is never done in the
> > > > kernel, but is a natural
> > > > affect of the Watcom C compiler.  There's also strict rules about
> > > > register assignments that re enforced between assembler modules in
> > > > NetWare to reduce the overhead of a context switch.  The code path is
> > > > very complex in NetWare, and priorities and all this stuff exists, but
> > > > these code paths are segragated so these types of checks only happen
> > > > once in a while and check a pre-calc'd "scoreboard" that is read only
> > > > across processors and updated and recal'd by a timer every 18 ticks.
> > > >
> > > > Jeff
> > > >
> > > >
> > > >
> > > > Andi Kleen wrote:
> > > > >
> > > > > On Tue, Oct 31, 2000 at 02:52:11PM -0700, Jeff V. Merkey wrote:
> > > > > > The numbers don't lie.  You know where the code is.  You notice that
> > > > > > there is a version of
> > > > > > the kernel hand coded in assembly language.  You'l also noticed that
> > > > > > it's SMP and takes ZERO LOCKS during context switching, in fact, most of
> > > > > > the design is completely lockless.
> > > > >
> > > > > I suspect most of the confusion in this thread comes because you seem to
> > > > > use a different definition of context switch than Ingo and others. Could
> > > > > you explain what you exactly mean with a context switch ?
> > > > >
> > > > > -Andi
> > > -
> > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > > the body of a message to majordomo@vger.kernel.org
> > > Please read the FAQ at http://www.tux.org/lkml/
> > >
> > 
> > -----BEGIN PGP SIGNATURE-----
> > Version: PGP 6.5.2
> > 
> > iQEVAwUBOf9LYj7msCGEppcbAQFdZQgAocWtMwnNmmLnfSS+cGHZd8V0IFfmoVb7
> > fDRoNWMmOzU5g5W8aAudQFqGpGqizBR8/AA9ziqHRfKxwoo5/nuHtEMDpfw0nV5e
> > ghsd7qtzv1kTk0l5zp6bN2qPlGgs7Ke72od10X6pTGDyuUDQK71YNQ9UUcCv8GEO
> > 2PpPOnCHw3atuQ0hetMNcFfIdvvslTB2+pcVzYxWWGhCYIeWreF8w1qf8XDYQil9
> > Ih22vmu69LP03RwXkFioikVSK8F8m+31DUBA67exN+R4qXy8+U5ZtyPQ+onIeguh
> > SnADBNjGjWK2mPyLNSVyAwH6EsIaGzk1QJ5hYULzVFi4zVl3pyRi8w==
> > =91LL
> > -----END PGP SIGNATURE-----
> 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Larry,

What's your mailing address and I'll send you out a legally licensed
copy of NetWare 3.12 and transfer the license to you then you can do the
comparison and see for yourself.

:-)

Jeff

Larry McVoy wrote:
> 
> On Tue, Oct 31, 2000 at 03:47:56PM -0700, Jeff V. Merkey wrote:
> > It kicks Linux's but in LAN I/O scaling.
> 
> Really?  So, since in a few messages back you claimed that it has a fully
> supported userland which implements all of P1003.1 as well as sockets,
> obviously, since it is a networking operating system, it should be the
> work of a few minutes to download LMbench, compile it, and come back with
> the lat_tcp performance numbers which "kicks Linux's but".  Please do so.
> --
> ---
> Larry McVoy              lm at bitmover.com           http://www.bitmover.com/lm
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Larry McVoy]

On Tue, Oct 31, 2000 at 03:47:56PM -0700, Jeff V. Merkey wrote:
> It kicks Linux's but in LAN I/O scaling.  

Really?  So, since in a few messages back you claimed that it has a fully
supported userland which implements all of P1003.1 as well as sockets,
obviously, since it is a networking operating system, it should be the 
work of a few minutes to download LMbench, compile it, and come back with
the lat_tcp performance numbers which "kicks Linux's but".  Please do so.
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

David Lang wrote:
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> 
> Jeff, one other thing. Linux is not x86 hand-crafted assembler, it's
> capable of running on many platforms. are you planning on giving up this
> capability or hand crafting the kernel for each chip?
> 
> Linux on x86 is nice (and I do use it a lot) but one of the things that
> makes it so useful is that when you do outgrow what youcan do on a x86
> platform you cna move to a more powerful platform without having to change
> to a different OS.

How about a NetWare like NOS with all the application support of Linux? 
I think a lot of people will love it.  Novell's largest customers have
told me they want to see it, and would deploy it.  I do believe there's
a market for such a beast.  A very lucrative one...

Jeff

> 
>  David Lang
> 
> On Tue, 31 Oct 2000,
> Jeff V. Merkey wrote:
> 
> > Date: Tue, 31 Oct 2000 15:45:45 -0700
> > From: Jeff V. Merkey <jmerkey@timpanogas.org>
> > To: Andi Kleen <ak@suse.de>, mingo@elte.hu, Pavel Machek <pavel@suse.cz>,
> >      linux-kernel@vger.kernel.org
> > Subject: Re: 2.2.18Pre Lan Performance Rocks!
> >
> >
> >
> > One more optimization it has.  NetWare never "calls" functions in the
> > kernel.  There's a template of register assignments in between kernel
> > modules that's very strict (esi contains a WTD head, edi has the target
> > thread, etc.) and all function calls are jumps in a linear space.
> > layout of all functions are 16 bytes aligned for speed, and all
> > arguments in kernel are passed via registers.  It's a level of
> > optimization no C compiler does -- all of it was done by hand, and most
> > function s in fast paths are layed out in 512 byte chunks to increases
> > speed.  Stack memory activity in the NetWare kernel is almost
> > non-existent in almost all of the "fast paths"
> >
> > Jeff
> >
> > "Jeff V. Merkey" wrote:
> > >
> > > A "context" is usually assued to be a "stack".  The simplest of all
> > > context switches
> > > is:
> > >
> > >    mov    x, esp
> > >    mov    esp, y
> > >
> > > A context switch can be as short as two instructions, or as big as a TSS
> > > with CR3 hardware switching,
> > >
> > > i.e.
> > >
> > >    ltr    ax
> > >    jmp    task_gate
> > >
> > > (500 clocks later)
> > >
> > >    ts->eip gets exec'd
> > >
> > > you can also have a context switch that does an int X where X is a gate
> > > or TSS.
> > >
> > > you can also have a context switch (like linux) that does
> > >
> > >     mov    x, esp
> > >     mov    esp, y
> > >     mov    z, CR3
> > >     mov    CR3, w
> > >
> > > etc.
> > >
> > > In NetWare, a context switch is an in-line assembly language macro that
> > > is 2 instructions long for a stack switch and 4 instructions for a CR3
> > > reload -- this is a lot shorter than Linux.
> > > Only EBX, EBP, ESI, and EDI are saved and this is never done in the
> > > kernel, but is a natural
> > > affect of the Watcom C compiler.  There's also strict rules about
> > > register assignments that re enforced between assembler modules in
> > > NetWare to reduce the overhead of a context switch.  The code path is
> > > very complex in NetWare, and priorities and all this stuff exists, but
> > > these code paths are segragated so these types of checks only happen
> > > once in a while and check a pre-calc'd "scoreboard" that is read only
> > > across processors and updated and recal'd by a timer every 18 ticks.
> > >
> > > Jeff
> > >
> > >
> > >
> > > Andi Kleen wrote:
> > > >
> > > > On Tue, Oct 31, 2000 at 02:52:11PM -0700, Jeff V. Merkey wrote:
> > > > > The numbers don't lie.  You know where the code is.  You notice that
> > > > > there is a version of
> > > > > the kernel hand coded in assembly language.  You'l also noticed that
> > > > > it's SMP and takes ZERO LOCKS during context switching, in fact, most of
> > > > > the design is completely lockless.
> > > >
> > > > I suspect most of the confusion in this thread comes because you seem to
> > > > use a different definition of context switch than Ingo and others. Could
> > > > you explain what you exactly mean with a context switch ?
> > > >
> > > > -Andi
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@vger.kernel.org
> > Please read the FAQ at http://www.tux.org/lkml/
> >
> 
> -----BEGIN PGP SIGNATURE-----
> Version: PGP 6.5.2
> 
> iQEVAwUBOf9LYj7msCGEppcbAQFdZQgAocWtMwnNmmLnfSS+cGHZd8V0IFfmoVb7
> fDRoNWMmOzU5g5W8aAudQFqGpGqizBR8/AA9ziqHRfKxwoo5/nuHtEMDpfw0nV5e
> ghsd7qtzv1kTk0l5zp6bN2qPlGgs7Ke72od10X6pTGDyuUDQK71YNQ9UUcCv8GEO
> 2PpPOnCHw3atuQ0hetMNcFfIdvvslTB2+pcVzYxWWGhCYIeWreF8w1qf8XDYQil9
> Ih22vmu69LP03RwXkFioikVSK8F8m+31DUBA67exN+R4qXy8+U5ZtyPQ+onIeguh
> SnADBNjGjWK2mPyLNSVyAwH6EsIaGzk1QJ5hYULzVFi4zVl3pyRi8w==
> =91LL
> -----END PGP SIGNATURE-----
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Ingo Molnar wrote:
> 
> On Tue, 31 Oct 2000, Jeff V. Merkey wrote:
> 
> > One more optimization it has.  NetWare never "calls" functions in the
> > kernel.  There's a template of register assignments in between kernel
> > modules that's very strict (esi contains a WTD head, edi has the target
> > thread, etc.) and all function calls are jumps in a linear space.
> 
> this might be a win on a i486, but is a loss with any branch predicting,
> large-pipeline CPUs (think Pentium IV), which are optimized for CALLs, not
> for JMP *EAX instructions. This is the problem with assembly optimizations
> that try to compete with the compiler's work: hand-made assembly can only
> get worse over time (stay constant in the best case), while compilers are
> known to improve slowly but steadily. Plus hand-made assembly is a huge
> stone tied to your legs if you try to swim to other architectures. Eg. we
> quite often make use of GCC's register-based function parameter passing
> optimization. We do use hand-made assembly in a number of cases in Linux
> as well, and double-check GCC's assembly output in critical code paths,
> but we try to not make it an essential facility.

It's hand optimized for all these cases to one of the highest degrees
that exist anywhere in the world.   Intel and Novell have been in bed
together for a very long time...

Jeff

> 
>         Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Michael H. Warfield]

On Tue, Oct 31, 2000 at 02:52:11PM -0700, Jeff V. Merkey wrote:

> Ingo Molnar wrote:
> > 
> > On Tue, 31 Oct 2000, Pavel Machek wrote:
> > 
> > > > Excuse me, 857,000,000 instructions executed and 460,000,000
> > > > context switches a second -- on a PII system at 350 Mhz. [...]
> > 
> > > That's more than one context switch per clock. I do not think so.
> > > Really go and check those numbers.
> > 
> > yep, you cannot have 460 million context switches on that system,
> > unless you have some Clintonesque definition for 'context switch' ;-)

> The numbers don't lie.  You know where the code is.  You notice that
> there is a version of
> the kernel hand coded in assembly language.  You'l also noticed that
> it's SMP and takes ZERO LOCKS during context switching, in fact, most of
> the design is completely lockless.

	Ah ha ha ha!!!!  Sure they do!  You're just quoting statistics
measured under whatever conditions you imposed.

	Numbers lying?  I think the famous line has been variously
attributed to either Mark Twain or Disraeli (don't know which really
coined the phrase) but it's been said that there are three kinds of
lies "Lies, Damn Lies, and Statistics".  Yes numbers do lie.  Sometimes
it's the GIGO law and sometimes its just the fact that if you abuse
statistics long enough they will tell you ANYTHING.  Sometimes it merely
the person manipulating^H^H^H^H^H^H^H^H^H^H^H^Hproviding the numbers.

	BTW...  I was going to stay OUT of this rat trap, but since I'm
in for a dime, I might as well be in for a dolar...

	<Minor Rant>

	Comparisons have been made between the performance of Linux with
early (3.x, 4.x) versions of Novell.  ANYONE who wants to compare Linux
with that bug ridden, unreliable swamp of headaches and security holes
(somewhere in my archives I have the virus launcher that BYPASSES the
3.x login program) should be beaten about the head with a good textbook
on reliable coding techniques.  Novell made its hayday by beating the
bejesus out of TCP/IP and others primarily by disabling checksumming,
memory protection, and other reliability techniques.  Yes, they got
better performance on low performace processors, but at what cost?  Now
we cover their performance with reliability and superior hardware.  I
remember one misguided soul wanting to run IPX over SLIP pleading for help
on the Novell mailing list years ago.  Let's see...  SLIP eliminates the
MAC layer checksumming and IPX eliminates the error checking on the next
layer up...  Yup...  There was a receipe for random acts of terrorism.
Now we have PPP (this was in the days BEFORE PPP) and you could do it.
IPX depended on the lower layers for data integrity and and SLIP depended
on the layer above it.  Ooooppppssss....

	Then we had the Novel 5.x NFS server that allow you to create
scripts that were SUID to root just by making them read only to the
Novell workstations (ok - that's not performace related - I just think
that security should be given a LITTLE thought).  I worked at an outfit
(Syntrex) that saw themselves as becoming the "K-Mart of Novell" and I
was told that Novell was the be all and end all of networking and there
was really no future in this antiquated TCP/IP stuff.  I was laid off and
given all sorts of nice neat little toys like an AT&T source license
because they saw no future in Unix or TCP/IP.  (Bitter - no...  I have
had my revenge in spades...  They had no clue what they gave away and
let slip through their fingers!  :-) )

	Now, Novell has been dragged kicking and screaming into the TCP/IP
world, and Novell has been forced to add memory protection (at a performace
cost) to their servers, and the outfit that thought TCP/IP was history
is now history (Syntrex went Chapter 13 about 10 years ago), and I've had
the pleasure of slamming one particularly simple minded Novell rep (another
ex Syntrex inDUHvidual) with more than one security hole (the perl module
on the Novell web server was an absolute classic).

	My point here is that packets per second don't mean jack shit if
you can't do it reliably and you can't do it securely.  Novell failed on
both of those counts and those are a contributing factor in their current
troubles.  They built their reputation on performance that was achieved at
the expense of reliability and security.  Now they have to play with the
big boys and all the nasty kiddies out there who don't play nice.

	Performance is important.  Performance is desirable.  Efforts to
improve performance are worthwhile.  But performance should NEVER come at
the cost of security or reliability or integrity.  Comparisions with high
performance systems which lacked security, reliability, and data integrity
are suspect AT BEST.  We should NEVER give up the quest for better
performance but comparisons to an inferior operating system which can pump
out packets faster than us is not the threat some people would like it to be.

	</Minor Rant>

	My regards and respects to Jeff.  He says he was responsible for the
Novell 4.x and 5.x systems.  I note that he omitted the 3.x OS.  Acknowledged
and respected!  In my earlier days, I was the kernel jock responsible for a
proprietary version of XENIX and worked on Microport UNIX (may someone
forever drive a stake through that bastard's heart) and SCO Unix.  I'm
"contaminated goods" for certain projects so I can't contribute to certain
applications like Taylor UUCP because I have legal source code to HDB UUCP
(as if UUCP means jack in today world).  Does the current taylor UUCP have
the most possible efficient checksumming algorithm for the UUCP 'g' protocol?
No...  No way...  I've seen one that stomps Taylor UUCP's ass and takes names
from the AT&T SVR5 release 3.2 source tapes (took me a week to figure out just
how it worked - damn those comment strippers).  Does it matter?  Not one bit.

	I want to see Linux excel.  Does that mean that it has to beat
every benchmark set by an operating system that cut every corner that
would cause Linus to turn into a Quake Balrog?  I think not.

> Jeff

> >         Ingo

	Mike
-- 
 Michael H. Warfield    |  (770) 985-6132   |  mhw@WittsEnd.com
  (The Mad Wizard)      |  (678) 463-0932   |  http://www.wittsend.com/mhw/
  NIC whois:  MHW9      |  An optimist believes we live in the best of all
 PGP Key: 0xDF1DD471    |  possible worlds.  A pessimist is sure of it!

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Alan Cox]

> One more optimization it has.  NetWare never "calls" functions in the
> kernel.  There's a template of register assignments in between kernel
> modules that's very strict (esi contains a WTD head, edi has the target
> thread, etc.) and all function calls are jumps in a linear space. 

What if I jump to an invalid address - does it crash ?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Alan Cox wrote:
> 
> > One more optimization it has.  NetWare never "calls" functions in the
> > kernel.  There's a template of register assignments in between kernel
> > modules that's very strict (esi contains a WTD head, edi has the target
> > thread, etc.) and all function calls are jumps in a linear space.
> 
> What if I jump to an invalid address - does it crash ?

The jumps are all to defined labels in the code, so there's never one
that's invalid.  The only exception are the jump tables in the MSM and
TSM lan modules, and yes it does crash is a bad driver gets loaded, but
that's the same as Linux.  NetWare avoids using jump tables since they
cause an AGI to get generated, which will interlock the processor
pipelines.  If you load an address on Intel, then immediately attempt to
use it, the processor will generate and Address Generation Interlock
(AGI) that will interlock the piplines so the request can be serviced
immediately.  This cost 2 clocks per address jump used, plus it shuts
down the paralellism in the piplines.

Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Richard B. Johnson]

On Tue, 31 Oct 2000, Jeff V. Merkey wrote:

> 
> A "context" is usually assued to be a "stack".  The simplest of all
> context switches 
> is:
> 
>    mov    x, esp
>    mov    esp, y
> 
> A context switch can be as short as two instructions, or as big as a TSS
> with CR3 hardware switching,
> 
> i.e.  
> 
>    ltr    ax
>    jmp    task_gate
> 
> (500 clocks later)
> 
>    ts->eip gets exec'd
> 
> you can also have a context switch that does an int X where X is a gate
> or TSS.
> 
> you can also have a context switch (like linux) that does
> 
>     mov    x, esp
>     mov    esp, y
>     mov    z, CR3
>     mov    CR3, w
> 
> etc.
> 
> In NetWare, a context switch is an in-line assembly language macro that
> is 2 instructions long for a stack switch and 4 instructions for a CR3
> reload -- this is a lot shorter than Linux.
> Only EBX, EBP, ESI, and EDI are saved and this is never done in the
> kernel, but is a natural
> affect of the Watcom C compiler.  There's also strict rules about
> register assignments that re enforced between assembler modules in
> NetWare to reduce the overhead of a context switch.  The code path is
> very complex in NetWare, and priorities and all this stuff exists, but
> these code paths are segragated so these types of checks only happen
> once in a while and check a pre-calc'd "scoreboard" that is read only
> across processors and updated and recal'd by a timer every 18 ticks.
> 
> Jeff
> 
>    

I have this feeling that this is an April Fools joke. Unfortunately
it's Halloween.

One could create a 'kernel' that does:
	for(;;)
        {
          proc0();
          proc1();
          proc2();
          proc3();
          etc();
        }

... and loop forever. All 'tasks' would just be procedures and no
context-switching would even be necessary. This is how some network
file-servers worked in the past (Vines comes to mind). Since all
possible 'tasks' are known at compile-time, there isn't even any
need for memory protection because every task cooperates and doesn't
destroy anything that it doesn't own.

The only time you need to save anything is for interrupt handlers.
This was some simple push/pops of only the registers actually
used in the ISR.

Now, the above example may seem absurd, however it's not. Inside
each of the proc()'s is a global state-variable that allows the
code to start executing at the place it left off the last time
through. If the code was written in 'C' it would be a 'switch'
statement. The state-variable for each of the procedures is global
and can be changed in an interrupt-service-routine. This allows
interrupts to change the state of the state-machines.

This kind of 'kernel' is very fast and very effective for things
like file-servers and routers, things that do the same stuff over
and over again.

However, there techniques are not useful with a kernel that
has an unknown number of tasks that execute 'programs' that are
not known to the kernel at compile-time, such as a desk-top
operating system.

These operating systems require context-switching. This requires
that every register that a user could possibly alter, be saved
and restored. It also requires that the state of any hardware
that a user could be using, also be save and restored. This
cannot be done in 2 instructions as stated. Further, this saving
and restoring cannot be a side-effect of a particular compiler, as
stated.

Cheers,
Dick Johnson

Penguin : Linux version 2.2.17 on an i686 machine (801.18 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Tue, 31 Oct 2000, Jeff V. Merkey wrote:

> > > > Excuse me, 857,000,000 instructions executed and 460,000,000
> > > > context switches a second -- on a PII system at 350 Mhz. [...]
> > 
> > > That's more than one context switch per clock. I do not think so.
> > > Really go and check those numbers.
> > 
> > yep, you cannot have 460 million context switches on that system,
> > unless you have some Clintonesque definition for 'context switch' ;-)
> 
> The numbers don't lie. [...]

sure ;) I can do infinite context switches! You dont believe? See:

	#define schedule() do { } while (0)

[there is a small restriction, should only be used in single-task
systems.]

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

"Richard B. Johnson" wrote:
> 

Dick,

In NetWare this:

> 
> One could create a 'kernel' that does:
>         for(;;)
>         {
>           proc0();
>           proc1();
>           proc2();
>           proc3();
>           etc();
>         }

would be coded like this (no C compiler):

proc0:

proc1:

proc2:

proc3:

etc:

label:
     jmp  proc0


I just avoided 5 x 20 bytes of pushes and pops on the stack ad optimized
for a simple fall through case.  

:-)

Jeff

> 
> Cheers,
> Dick Johnson
> 
> Penguin : Linux version 2.2.17 on an i686 machine (801.18 BogoMips).
> 
> "Memory is like gasoline. You use it up when you are running. Of
> course you get it all back when you reboot..."; Actual explanation
> obtained from the Micro$oft help desk.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Ingo Molnar wrote:
> 
> On Tue, 31 Oct 2000, Jeff V. Merkey wrote:
> 
> > [...] These types of optimizations are possible when people have
> > acccess to Intel Red Cover documents, [...]
> 
> optimizing away AGIs has been documented by public Intel PDFs for years:
> 
>  [...] Since the Pentium processor has two integer pipelines, a register
>  used as the base or index component of an effective address calculation
>  (in either pipe) causes an additional clock cycle if that register is the
>  destination of either instruction from the immediately preceding clock
>  cycle. This effect is known as Address Generation Interlock (AGI).
> 
> (ditto for the p6 core CPUs), and GCC (IIRC) tries to avoid AGI conflicts
> as much as possible. (and this kind of stuff belongs into the compiler)

Odd.  When I profile Linux with EMON, I see tons of them.  Anywhere code
does

mov    eax, addr
mov    [addr], ebx

one will get generated.  Even something as simple as:

mov    eax, addr
inc    eax
dec    eax
mov    [addr]. ebx

will avoid an AGI (since the other pipeline has now been allowed a few
clocks to fetch the address and load it).

Jeff



> 
>         Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Ingo Molnar wrote:
> 
> On Tue, 31 Oct 2000, Jeff V. Merkey wrote:
> 
> > > One could create a 'kernel' that does:
> > >         for(;;)
> > >         {
> > >           proc0();
> > >           proc1();
> > >           proc2();
> > >           proc3();
> > >           etc();
> > >         }
> >
> > would be coded like this (no C compiler):
> >
> > proc0:
> >
> > proc1:
> >
> > proc2:
> >
> > proc3:
> >
> > etc:
> >
> > label:
> >      jmp  proc0
> 
> oh, and what happens if it turns out that some other place wants to call
> proc3 as well? Recode the assembly - cool! Not.

They would load the registers to the proper values and jump to it.  The
return address would be stored in the ESI register, and when the routie
completed, it would do

jmp   esi  

to return, with 0 stack usage.

> 
> > I just avoided 5 x 20 bytes of pushes and pops on the stack ad optimized
> > for a simple fall through case.
> 
> FYI, GCC does not generate 5 x 20 bytes of pushes and pops. In fact in the
> above specific case it will not generate a single push (automatically -
> you dont have to worry about it).

If the compiler were set to optimize.  

Jeff

> 
>         Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Roger Larsson]

"Jeff V. Merkey" wrote:
> 
> David/Alan,
> 
> Andre Hedrick is now the CTO of TRG and Chief Scientist over Linux
> Development.  After talking
> to him, we are going to do our own ring 0 2.4 and 2.2.x code bases for
> the MANOS merge.
> the uClinux is interesting, but I agree is limited.
> 

Jeff,

What would be missed out in this approach:
* Use Montavista "fully" preemtible kernel.
* Using Kernel threads for all services (File, Print, Web, etc.).

/RogerL

--
Home page:
  http://www.norran.net/nra02596/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Tue, 31 Oct 2000, Jeff V. Merkey wrote:

> A "context" is usually assued to be a "stack". [...]

a very clintonesque definition indeed ;-)

what is relevant is the latency to switch from one process to another one.
And this is what we call a context switch. It includes scheduling
decisions and all sorts of other stuff. You are comparing stack &
caller-saved register switching performance (which is just a small part of
context switching and has no standing alone) against full Linux context
switch performance (this is what i quoted), and thus you have won my
'Mindcraft benchmark of the day' award :-)

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Tue, 31 Oct 2000, Jeff V. Merkey wrote:

> One more optimization it has.  NetWare never "calls" functions in the
> kernel.  There's a template of register assignments in between kernel
> modules that's very strict (esi contains a WTD head, edi has the target
> thread, etc.) and all function calls are jumps in a linear space. 

this might be a win on a i486, but is a loss with any branch predicting,
large-pipeline CPUs (think Pentium IV), which are optimized for CALLs, not
for JMP *EAX instructions. This is the problem with assembly optimizations
that try to compete with the compiler's work: hand-made assembly can only
get worse over time (stay constant in the best case), while compilers are
known to improve slowly but steadily. Plus hand-made assembly is a huge
stone tied to your legs if you try to swim to other architectures. Eg. we
quite often make use of GCC's register-based function parameter passing
optimization. We do use hand-made assembly in a number of cases in Linux
as well, and double-check GCC's assembly output in critical code paths,
but we try to not make it an essential facility.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Tue, 31 Oct 2000, Jeff V. Merkey wrote:

> It kicks Linux's but in LAN I/O scaling. [...]

brain cacheflush? Restart the same thread? Sorry i've got better things to
do.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Tue, 31 Oct 2000, Jeff V. Merkey wrote:

> [...] These types of optimizations are possible when people have
> acccess to Intel Red Cover documents, [...]

optimizing away AGIs has been documented by public Intel PDFs for years:

 [...] Since the Pentium processor has two integer pipelines, a register
 used as the base or index component of an effective address calculation
 (in either pipe) causes an additional clock cycle if that register is the
 destination of either instruction from the immediately preceding clock
 cycle. This effect is known as Address Generation Interlock (AGI).

(ditto for the p6 core CPUs), and GCC (IIRC) tries to avoid AGI conflicts
as much as possible. (and this kind of stuff belongs into the compiler)

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Tue, 31 Oct 2000, Jeff V. Merkey wrote:

> > One could create a 'kernel' that does:
> >         for(;;)
> >         {
> >           proc0();
> >           proc1();
> >           proc2();
> >           proc3();
> >           etc();
> >         }
> 
> would be coded like this (no C compiler):
> 
> proc0:
> 
> proc1:
> 
> proc2:
> 
> proc3:
> 
> etc:
> 
> label:
>      jmp  proc0

oh, and what happens if it turns out that some other place wants to call
proc3 as well? Recode the assembly - cool! Not.

> I just avoided 5 x 20 bytes of pushes and pops on the stack ad optimized
> for a simple fall through case.  

FYI, GCC does not generate 5 x 20 bytes of pushes and pops. In fact in the
above specific case it will not generate a single push (automatically -
you dont have to worry about it).

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Tue, 31 Oct 2000, Jeff V. Merkey wrote:

> Odd.  When I profile Linux with EMON, I see tons of them.  Anywhere code
> does
> 
> mov    eax, addr
> mov    [addr], ebx

AGIs were a real problem on P5 class Intel CPUs. On P6 core CPUs, most
forms of addresses (except memory writes) do not generate any AGIs. And
the AGI on P6 cores does not keep up the pipeline, unless you reuse the
same address. (which would be stupid in most cases) I bet Crusoe's have no
AGIs at all. Do you see the trend in CPU design?

	Ingo



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Ingo Molnar]

On Tue, 31 Oct 2000, Richard B. Johnson wrote:

> However, these techniques are not useful with a kernel that has an
> unknown number of tasks that execute 'programs' that are not known to
> the kernel at compile-time, such as a desk-top operating system.

yep, exactly. It simply optimizes the wrong thing and restricts
architectural flexibility. It is very easy to optimize by making
a system more specific. (this is fact is a more or less automatic
engineering work) The real optimizations are the ones that do not
take away from the generic nature of the system.

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Davide Libenzi]

On Wed, 01 Nov 2000, Jeff V. Merkey wrote:
> 
> mov    eax, addr
> mov    [addr], ebx
> 

Probably You mean this :

mov	r/imm, %eax
mov	(%eax), %ebx


- Davide
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Horst von Brand]

Jesse Pollard <pollard@tomcat.admin.navo.hpc.mil> said:

[...]

> Also pay attention to the security aspects of a true "zero copy" TCP stack.
> It means that SOMETIMES a user buffer will recieve data that is destined
> for a different process.

Why? AFAIKS, given proper handling of the issues involved, this can't
happen (sure can get tricky, but can be done in principle. Or am I
off-base?)
-- 
Horst von Brand                             vonbrand@sleipnir.valparaiso.cl
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Horst von Brand]

"Jeff V. Merkey" <jmerkey@timpanogas.org> said:
> One more optimization it has.  NetWare never "calls" functions in the
> kernel.  There's a template of register assignments in between kernel
> modules that's very strict (esi contains a WTD head, edi has the target
> thread, etc.) and all function calls are jumps in a linear space. 
> layout of all functions are 16 bytes aligned for speed, and all
> arguments in kernel are passed via registers.  It's a level of
> optimization no C compiler does -- all of it was done by hand, and most
> function s in fast paths are layed out in 512 byte chunks to increases
> speed.  Stack memory activity in the NetWare kernel is almost
> non-existent in almost all of the "fast paths"

Nice! Now run that (i386 optimized?) beast on a machine that works
different (latest K7s perhaps?), and many optimizations break.

When you got that fixed, would you please port it to Alpha?

Sure, using C (with a not-overly-bright compiler) has a non-negligible
cost. But huge benefits too. The whole of software (including OS) design is
an excercise in delicate juggling among conflicting goals. Had Linus gone
down the "all-assembler, bummed to its limits" route, Linux would have been
dead by 0.03 or so, depending on the stubborness of its creator to be sure.
-- 
Horst von Brand                             vonbrand@sleipnir.valparaiso.cl
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jesse Pollard]

On Tue, 31 Oct 2000, Horst von Brand wrote:
>Jesse Pollard <pollard@tomcat.admin.navo.hpc.mil> said:
>
>[...]
>
>> Also pay attention to the security aspects of a true "zero copy" TCP stack.
>> It means that SOMETIMES a user buffer will recieve data that is destined
>> for a different process.
>
>Why? AFAIKS, given proper handling of the issues involved, this can't
>happen (sure can get tricky, but can be done in principle. Or am I
>off-base?)

As I understand the current implementation, this can't. One of the optimizations
I had read about (for a linux test) used zero copy to/from user buffer as well
as zero copy in the kernel. I believe the DMA went directly to the users memory.

This causes a problem when/if there is a context switch before the data is
actually transferred to the proper location. The buffer isn't ready for use,
but could be examined by the user application (hence the security problem).

It was posed that this is not a problem IF the cluster (and it was a beowulf
cluster under discussion) is operated in a single user, dedicated mode.
In which case, to examine the buffer would either be a bug in the program,
or a debugger looking at a buffer directly.

To my knowlege, zero copy is only done to/from device and kernel. Userspace
has to go through a buffer copy (one into user space; one output from user
space) for all IP handling. All checksums are either done by the device,
or done without copying the data.

-- 
-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@cats-chateau.net

Any opinions expressed are solely my own.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Peter Samuelson]

    [Jeff Merkey]
> > > The numbers don't lie. [...]
> > 
  [Ingo Molnar]
> > sure ;) I can do infinite context switches! You dont believe? See:
> > 
> >         #define schedule() do { } while (0)

[Jeff]
> Actually, I think the compiler would optimize this statement
> completely out of the code.

That was Ingo's point.  He is doing infinite context switches per
second, where a context switch is defined as "schedule()", because he
turned it into a noop.

I.e. the numbers can easily be made to lie, by playing with the rules
of the game.

The point of confusion, Jeff, is that to *you* a context switch means
stack switch, with no baggage like scheduling or reloading registers.
Everyone else here thinks of a context switch as meaning a pre-emptive
switch between two unrelated processes -- which as you know involves
not only the stack, but MM adjustments, registers, floating point
registers (expensive on pre-P6), IP, and some form of scheduling.

Obviously some of these can be optimized out if you can make
assumptions about the processes: you might drop memory protection if
you like the stability of Windows 95, floating point if you can get
away with telling people they can't use it, maybe use FIFO scheduling
if you don't care about fairness and you know the processes are more or
less uniform.  Linux cannot make any of these assumptions -- it is far
too general-purpose.

In Linux, in fact, jumping from ring 3 to ring 0 (ie system call) is
not considered a context switch.  I suppose you would consider it one.
So the real question is, how many gettimeofday() per sec can Linux do?

Peter
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Larry McVoy]

On Tue, Oct 31, 2000 at 11:01:20PM -0600, Peter Samuelson wrote:
> So the real question is, how many gettimeofday() per sec can Linux do?

Oh, about 3,531,073 on a 1Ghz AM thunderbird running 
Linux disks.bitmover.com 2.4.0-test5.

That's 283.2 nanoseconds per call, to save you the math.
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Peter Samuelson]

  [me]
> > So the real question is, how many gettimeofday() per sec can Linux
> > do?

[Larry McVoy]
> Oh, about 3,531,073 on a 1Ghz AM thunderbird running 
> Linux disks.bitmover.com 2.4.0-test5.

So, at two "context switches" (Jeff's term) per syscall, we're
somewhere around half the speed of Netware's longjmp.

Not bad. (:

Peter
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Daniel Phillips]

"Jeff V. Merkey" wrote:
> A "context" is usually assued to be a "stack".  The simplest of all
> context switches is:
> 
>    mov    x, esp
>    mov    esp, y

Presumeably you'd immediately do a ret to some address, and there pop a
base address off the stack to get some global memory.  Is that right? 
Your context switches would be inline, and you'd have hardcoded which
process to execute next in most cases.

I'll buy the concept that changing stacks amounts to changing contexts,
so long as you follow certain rules.  Obviously, rules are what define a
context.  What are the two instructions that precede and the two
instructions that follow?  I'd guess, something like this:

   push bp
   push $1
   mov x, esp
   mov esp, y
   ret
$1 pop bp

--
Daniel
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: / on ramfs, possible?

[aer-list]

[-- Attachment #1: Type: text/plain, Size: 962 bytes --]


> On Mon, 30 Oct 2000, H. Peter Anvin wrote:
> 
> > Pardon?!  This doesn't make any sense...
> > 
> > The question was: how do switch from the initrd to using the ramfs as /? 
> > Using pivot_root should do it (after the pivot, you can of course nuke
> > the initrd ramdisk.)
> 
> My question is: What do you want to do that for? You can nuke the initrd
> ramdisk, but you can't drop the rd.c code, or ll_rw_blk.c code, etc. So
> why not just keep your root filesystem in the initrd where it started off?
> 

Because the stuff on the initrd is not at all what I want in the resultant filesystem. The machine(s) in question are some disk-based some disk-less. I have a system for remote installation/removal of packages (--> the "partition" _has_ to be able to grove/shrink). 

The package builder should not have to worry about these details, so the directory hierarcy cannot contain any traces of _how_ the packges got there in the first place.

/Anders 






[-- Attachment #2: Type: application/pgp-signature, Size: 235 bytes --]

Re: 2.2.18Pre Lan Performance Rocks!

[Horst von Brand]

Jesse Pollard <pollard@cats-chateau.net> said:
> On Tue, 31 Oct 2000, Horst von Brand wrote:
> >Jesse Pollard <pollard@tomcat.admin.navo.hpc.mil> said:
> >
> >[...]
> >
> >> Also pay attention to the security aspects of a true "zero copy" TCP stack.
> >> It means that SOMETIMES a user buffer will recieve data that is destined
> >> for a different process.

> >Why? AFAIKS, given proper handling of the issues involved, this can't
> >happen (sure can get tricky, but can be done in principle. Or am I
> >off-base?)

> As I understand the current implementation, this can't. One of the
> optimization s I had read about (for a linux test) used zero copy to/from
> user buffer as well as zero copy in the kernel. I believe the DMA went
> directly to the users memory.

Right. This means you have to ensure (somehow blocking the process(es) with
access to the buffer(s) involved) that nobody can see half-filled buffers.
Tricky, but not impossible, at least not in principle. Or play VM games and
switch the areas underneath atomically. The VM games we have been told are
costlier than the average copy on "typical" machines (PCs, presumably ;-),
plus you'd have to either ensure aligned buffers (how?) or keep two copies
of whatever surrounds them (where is the advantage then?).
-- 
Horst von Brand                             vonbrand@sleipnir.valparaiso.cl
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

"Jeff V. Merkey" wrote:

In the example of an AGI generating code fragment, while I described the
sequence of creating the AGI with immediate address usage correctly
(i.e. if you load an address into a register then immediately attempt to
use it, it will generate an AGI), I failed to put the register in the
coding example.  

A couple of folks are testing the gcc compiler for AGI problems as a
result of this post, and I am posting the corrected code for their
tests.

This code fragment will generate an AGI condition:

mov   eax, addr
mov   [eax].offset, ebx

You can do it with any register combination, BTW, eax and abx are
provided as examples.  For those who are monitoring the code produced by
gcc, this is the example to use to test generate an AGI correctly.

:-)

Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Jeff V. Merkey]

Davide Libenzi wrote:
> 
> On Thu, 02 Nov 2000, Jeff V. Merkey wrote:
> > "Jeff V. Merkey" wrote:
> > This code fragment will generate an AGI condition:
> >
> > mov   eax, addr
> > mov   [eax].offset, ebx
> 
> I had already posted the correction.
> It was clear that You had forgot something coz Your old code fragment did not
> generate AGI.

typing too fast late at night.  I described it correctly, I just forgot
to add the register indirection.  

:-)

Jeff

> 
> - Davide
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Davide Libenzi]

On Thu, 02 Nov 2000, Jeff V. Merkey wrote:
> "Jeff V. Merkey" wrote:
> This code fragment will generate an AGI condition:
> 
> mov   eax, addr
> mov   [eax].offset, ebx

I had already posted the correction.
It was clear that You had forgot something coz Your old code fragment did not
generate AGI.


- Davide
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: 2.2.18Pre Lan Performance Rocks!

[Daniel Phillips]

"Jeff V. Merkey" wrote:
> A "context" is usually assued to be a "stack".  The simplest of all
> context switches is:
> 
>    mov    x, esp
>    mov    esp, y

Is that your two instruction context switch?  The problem is, it doesn't
transfer control anywhere.  Maybe it doesn't need to.  I guess you could
break your tasks up into lots of little chunks and compile each chunk
inline and use actual calls to take you off the fast path.  The stack
changes are actually doing some useful work here: you might for instance
be processing a network packet whose address is on the stack.  But
somehow I don't think this is your two-instruction context switch.  The
only halfway flexible two-instruction context switch I can think of is:

    mov esp, y
    ret

where you already know the stack depth where you are so you don't have
to store it, and the task execution order is predetermined.  This
switches the *two* essential ingredients of a context: control+data. 
But there's a big fat AGI there and all the overhead of a jump so it
doesn't get your superscalar performance.

Now my stupid question: why on earth do you need a billion context
switches a second?

--
Daniel
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[tytso]

Travel and other things have kept me more than a little busy lately, so
I fell a bit behind during the test10-pre* series, and didn't have a
chance to issue new updates of the 2.4 TODO list as often as I would
like.  I've caught up on my backlog, however, and this is as up to date
as I can make it as of 2.4.0-test10.

As always, the list isn't perfect.  Comments and reports of bugs fixed /
not fixed are welcome.  (Although for my sanity, please change the
subject line before replying.  :-)

						- Ted


                         Linux 2.4 Status/TODO Page

   This list is almost always out of date, by definition, since kernel
   development moves so quickly. I try to keep it as up to date as
   possible, though. Please send updates to tytso@mit.edu.

   Every few days or so, I periodically send updated versions of this
   list to the linux-kernel list, but you should consult
   http://linux24.sourceforge.net to get the latest information.

   If you're curious to see what has changed recently, check out
   http://linux24.sourceforge.net/status-changes.html. The previous set
   of changes can be found here. Also, this html file is managed under
   CVS at SourceForge.

   I try to keep e-mail addresses out of this document, since I don't
   want to make life easy for bottom-feeder spam artists. If you are a
   developer and want to contact the person who originally reported the
   problem, or want to see the e-mail message which prompted me to
   include a bug/issue in this list, contact me. I keep an mail archive
   which will have that information assuming it was an item added since I
   took over the list from Alan.

   Last modified: [tytso:20001103.1002EST]

   Hopefully up to date as of: test10

1. Should Be Fixed (Confirmation Wanted)

     * Fbcon races (cursor problems when running continual streaming
       output mixed with printk + races when switching from X while doing
       continuous rapid printing --- Alan)
     * USB: system hang with USB audio driver {CRITICAL} (David
       Woodhouse, Randy Dunlap, Narayan Desai) (Fixed with usb-uhci;
       uhci-alt is unknown -- randy dunlap)
     * USB: fix setting urb->dev in printer, acm, bluetooth, all serial
       drivers (Greg KH) {CRITICAL} (test10-pre1)
     * USB: race conditions on devices in use and being unplugged
       (test10)
     * USB: printer Device ID string should not be static; printers can
       update it (test10)
     * USB: fix usbdevfs memset() on IOC_WRITE (Dan Streetman) {CRITICAL}
       (test10)
     * USB: fix hub driver allocation/usage of portstr & tempstr (D.
       Brownell) (causes oops and memory corruptoin) {CRITICAL} (test10)
     * USB: USB mouse stopped working (2.4.0-test9) (Gunther Mayer)
       (fixed in test10)
     * USB: SMP/concurrent/thread-safe for scanner.c, mdc800.c, rio800.c
       (test10)
     * USB: printer open should not fail on printer not ready; add
       GETSTATUS ioctl (2.4.0-test10)

2. Capable Of Corrupting Your FS/data

     * Use PCI DMA by default in IDE is unsafe (must not do so on via
       VPx, x < 3) (Vojtech Pavlik --- requires chipset tuning to be
       enabled according to Andre Hedrick --- we need to turn this on by
       default, if it is safe -- TYT)
     * USB: Problems with USB storage drives (ORB, maybe Zip) during APM
       sleep/suspend
     * Bug in VFAT truncate code will cause slow and painful corruption
       of filesystem (Ivan Baldo, Alan Cox)

3. Security

     * Fix module remove race bug (still to be done: TTY, ldisc, I2C,
       video_device - Al Viro) (Rogier Wolff will handle ATM)

4. Boot Time Failures

     * Use PCI DMA 'lost interrupt' problem with PIIXn tuning enabled
       will hang laptop requiring physical power loss to restart (NEC
       Versa LX, 2.3.x to 2.4.0-test8-pre6, David Ford) (Vojtech Pavlik
       is looking at this)
     * Crashes on boot on some Compaqs ? (may be fixed)
     * Various Alpha's don't boot under 2.4.0-test9 (PCI resource
       allocation problem? Michal Jaegermann; Richard Henderson may have
       an idea what's failing.)
     * Palmax PD1100 hangs during boot since 2.4.0-test9 (Alan Cox)
     * Compaq proliant 7000 (with Compaq Smart Array-3100ES) hangs during
       2.4.0-test9. Likely related to the Raid driver given where it hung
       in the boot messages (chonga at isoft)

5. Compile errors

6. In Progress

     * Fix all remaining PCI code to use pci_enable_device (mostly done)
     * Finish the audit/code review of the code dealing with descriptor
       tables. (Al Viro)
     * DMFE is not SMP safe (Frank Davis patch exists, but hasn't gotten
       much commens yet)
     * Audit all char and block drivers to ensure they are safe with the
       2.3 locking - a lot of them are not especially on the
       read()/write() path. (Frank Davis --- moving slowly; if someone
       wants to help, contact Frank)
     * Fixing autofs4 to deal with VFS changes (Jeremy Fitzhardinge)

7. Obvious Projects For People (well if you have the hardware..)

     * Make syncppp use new ppp code
     * Fix SPX socket code

8. Fix Exists But Isnt Merged

     * Restore O_SYNC functionality (CRITICAL, DB's depend on it)
       (Stephen)
     * Update SGI VisWS to new-style IRQ handling (Ingo)
     * Support MP table above 1Gig (Ingo)
     * Dont panic on boot when meeting HP boxes with wacked APIC table
       numbering (AC)
     * Scheduler bugs in RT (Dimitris)
     * AIC7xxx doesnt work non PCI ? (Doug says OK, new version due
       anyway)
     * Fix boards with different TSC per CPU and kill TSC use on them
     * IRDA fixes:
          + Fixes from DAG: Incluldes a number of critical bugfixes.
            Detailed listing of changes here:
               o irda1
               o irda2
               o irda3
          + Fixes from Jean Tourrilhes (Fixes critical bugs: Infinite
            loop in /proc/discovery, unsafe discovery entry removal,
            potential out of array access in QoS handling) (Functional
            bugs fixed: Zombie sockets disable listening socket, some
            discovery acses unhandled)
     * Many network device drivers don't call MOD_INC_USE_COUNT in
       dev->open. (Paul Gortmaker has patches)
     * using ramfs with highmem enabled can yield a kernel NULL pointer
       dereference. (wollny at cns.mpg.de has a patch)
     * Writing past end of removeable device can cause data corruption
       bugs in the future (Jari Ruusu)
     * fix the UFS and sysvfs races (the latter couple is broken as ext2
       was, UFS is _completely_ broken; eats filesystems) (Al Viro --
       patch mostly exists)
     * HT6560/UMC8672 ide sets up stuff too early (before region stuff
       can be done) (Andre Herick has fix)
     * mtrr.c is broken for machines with >= 4GB of memory (David Wragg
       has a fix)

9. To Do

     * truncate->invalidate_inode_pages removes mapping information from
       mapped pages which may be dirty; sync_pte -> crash. (CRITICAL)
       (sct, Linus and AL are looking at this)
     * VM: raw I/O data loss (raw IO may arrive in a page which afer it
       is unammped from a process) (CRITICAL) (rik van riel)
     * Check all devices use resources properly (Everyone now has to use
       request_region and check the return since we no longer single
       thread driver inits in all module cases. Also memory regions are
       now requestable and a lot of old drivers dont know this yet. --
       Alan Cox)
     * Tulip hang on rmmod/crashes sometimes
     * Devfs races (mostly done - Al Viro)
     * Fix further NFS races (Al Viro)
     * Test other file systems on write
     * Fix mount failures due to copy_* user mishandling
     * Check all file systems are either LFS compliant or error large
       files
     * Issue with notifiers that try to deregister themselves? (lnz;
       notifier locking change by Garzik should backed out, according to
       Jeff)
     * Misc locking problems
          + drivers/pcmcia/ds.c: ds_read & ds_write. SMP locks are
            missing, on UP the sleep_on() use is unsafe.
          + Power management locking (Alan Cox)
          + USB:
               o USB: fix MOD_INC races in plusb.c and uss720.c
               o USB: fix concurrent read/write and other SMP like bugs
                 in:
                    # acm.c
                    # printer.c
                    # uss720.c
                    # serial/ftdi_sio.c
                    # serial/omninet.c
          + do_execve (Al Viro, reported by Manfred)
          + fix the quota races (Al Viro)
     * SCSI CD-ROM doesn't work on filesystems with < 2kb block size
       (Jens Axboe will fix)
     * Remove (now obsolete) checks for ->sb == NULL (Al Viro)
     * Audit list of drivers that dereference ioremap's return (Abramo
       Bagnara)
     * ISAPnP can reprogram active devices (2.4.0-test5, Elmer Joandi,
       alan)
     * Multilink PPP can get the kernel into a tight loop which spams the
       console and freezes the machine (Aaron Tiensivu)
     * mm->rss is modified in some places without holding the
       page_table_lock (sct)
     * Copying between two encrypting loop devices causes an immediate
       deadlock in the request queue (Andi Kleen)
     * FAT filesystem doesn't support 2kb sector sizes (did under 2.2.16,
       doesn't under 2.4.0test7. Kazu Makashima, alan)
     * The new hot plug PCI interface does not provide a method for
       passing the correct device name to cardmgr (David Hinds, alan)
     * non-PNP SB AWE32 has tobles in 2.4.0-test7 and newer (Gerard
       Sharp) (Paul Laufer has a potential patch)
     * 2.4.0-test8 pcmcia is unusable in fall forms (kernel, mixed, or
       dhinds code) (David Ford)
     * VGA Console can cause SMP deadlock when doing printk {CRITICAL}
       (Keith Owens)
     * Forwawrd port 2.2 fixes to allow 2 GHz or faster CPU's. {CRITICAL}
     * IDE tape driver broken; if the last data written does not make up
       a full block, then the driver won't be able to read back ANY part
       of the last block (Mikael Pettersson, Alan Cox)
     * VM: Fix the highmem deadlock, where the swapper cannot create low
       memory bounce buffers OR swap out low memory because it has
       consumed all resources {CRITICAL} (old bug, already reported in
       2.4.0test6)
     * VM: page->mapping->flush() callback in page_lauder() for easier
       integration with journaling filesystem and maybe the network
       filesystems
     * VM: maybe rebalance the swapper a bit... we do page aging now so
       maybe refill_inactive_scan() / shm_swap() and swap_out() need to
       be rebalanced a bit
     * DRM cannot use AGP support module when CONFIG_MODVERSIONS is
       defined (issue with get_module_symbol caused fix proposed by John
       Levon to be rejected)
     * Spin doing ioctls on a down netdeice as it unloads == BOOM
       (prumpf, Alan Cox) Possible other net driver SMP issues (andi
       kleen)
     * USB: SANE backend can't communicate to its scanner (sometimes,
       some scanners)
     * USB: OHCI memory corruption problem
     * USB: Fix differences in UHCI and OHCI HCD behaviors/semantics:
          + OHCI doesn't do URB timeouts
          + OHCI always does BULK_QUEUE (as David.B said, Bulk queueing
            is a UHCI notion; not needed in OHCI; fix not needed)
     * USB: fix USB_QUEUE_BULK problem in uhci.c (oopsen after a while)
       {CRITICAL}
     * USB: Fix serial/omninet.c to not require a small mtu setting in
       order to get a PPP link to work properly (as reported by Bernhard
       Reiter)
     * USB: pegasus: avoid warning spewage on disconnect
     * USB: OHCI optional zero length packet (USB_DISABLE_SPD at send)
     * USB: consistent short packet handling OHCI/UHCI (including
       0-length packets) (Roman)
     * USB: consistent URB next pointer handling by OHCI/UHCI (Roman)

10. To Do But Non Showstopper

     * Go through as 2.4pre kicks in and figure what we should mark
       obsolete for the final 2.4 (i.e. XT hard disk support?)
     * Union mount (Al Viro)
     * Per Process rtsigio limit
     * iget abuse in knfsd
     * ISAPnP IRQ handling failing on SB1000 + resource handling bug
     * Parallel ports should set SA_SHIRQ if PCI (eg in Plip)
     * Devfs compiled in but not mounted causes crap for ->mnt_devname of
       root (Al Viro)
     * PCMCIA/Cardbus hangs (Basically unusable - Hinds pcmcia code is
       reliable)
          + PCMCIA crashes on unloading pci_socket
     * ATM phy-chip-driver interface change for Firestream ATM card
       (Rogier Wolff)
     * Loop device can still hang (William Stearns has script that will
       hang 2.4.0-test7, Peter Enderborg has a short sequence that will
       hang 2.4.0test9)
     * USB: acm (modem) driver is slow compared to Windows drivers for
       same modems (maybe an HCD problem, not acm driver, or acm should
       use bulk queueing)
     * USB: add devfs support to drivers that don't have it
     * USB: add DocBook info to main USB driver interfaces (usb.c)
     * USB: Printer stalls at random places when printing large graphics.
       When printing big pictures (10..50 meg) the printer stalls
       halfway. The point where it stalls is random but the fact that it
       stalls is reproducable. Printing the same pictures using the
       parallel interface is ok. Printing text is ok anyway. (Frank van
       Maarseveen)
     * USB: Misc locking problems: fix concurrent read/write and other
       SMP-like bugs in:
          + bluetooth.c
          + serial/keyspan.c
          + serial/whiteheat.c
     * USB: fix usb_unlink_urb() bug when called with an urb that was
       used on a device that is no longer registered in the USB system.
     * USB: control pipe locking (mutual exclusion)
     * USB: use pci_alloc_consistent throughout (mostly OHCI; some people
       want UHCI also)
     * RTL 8139 cards sometimes stop responding. Both drivers don't
       handle this quite good enough yet. (reported by Rogier Wolff,
       tentatively reported as fixed by David Ford; reports from Frank
       Jacobberger and Shane Shrybman indicate that it doesn't appear to
       be fixed in test9)
     * tty_register_devfs and tty_unregister_devfs declare struct
       tty_struct as a local, which causes stack overflows for user-mode
       linux. (Jeff Dike)

11. To Check

     * Check O_APPEND atomicity bug fixing is complete
     * Protection on i_size (sct) [Al Viro mostly done]
     * Mikulas claims we need to fix the getblk/mark_buffer_uptodate
       thing for 2.3.x as well
     * VFS?VM - mmap/write deadlock (demo code seems to show lock is
       there)
     * kiobuf seperate lock functions/bounce/page_address fixes
     * Fix routing by fwmark
     * rw semaphores on inodes to fix read/truncate races ? [Probably
       fixed]
     * Not all device drivers are safe now the write inode lock isnt
       taken on write
     * Multiwrite IDE breaks on a disk error [minor issue at best]
       (hopefully fixed)
     * ACPI/APM suspend issue - IDE related stuff ? (requires full
       taskfile support that was vetoed by Linus)
     * NFS bugs are fixed
     * Chase reports of SMB not working
     * Some AWE cards are not being found by ISAPnP ??
     * RAM disk contents vanishing on cramfs (block change) and bforget
       cases
     * Disappointing performance of Software Raid, esp. write performance
       (reported by Nils Rennebarth)
     * List of potential problems found by Stanford students using g++
       hacks:
          + Andy Chou's list of mismatched spinlocks and interrupts/bh
            enable/disable.
          + Seth Andrew Hallem's list potentially sleeping functions
            called with interrupts off or spinlocks held.
          + Dawson Engler's list of potential kmalloc/kfree bugs
     * Potential races in file locking code (Christian Ehrhardt)
          + locks_verify_area checks the wrong range if O_APPEND is set
            and the current file position is not at the end of the file.
          + dito if the file position changes between the call to
            locks_verify_area and the actual read/write (requires a
            shared file pointer, an attacker can use this to circumvent
            virtually any mandatory lock).
          + active writes should prevent anyone from getting mandatory
            locks for the area beeing written.
          + active reads should prevent anyone from getting mandatory
            write locks for the area beeing read.
     * Possible race in b-tree code for HFS, HPFS, NTFS: insert into the
       tree whild doing a readdir (Matthew Wilcox)
     * Stressing the VM (IOPS SPEC SFS) with HIGHMEM turned on can hang
       system (linux-2.4.0test5, Ying Chen, Rik van Riel)
     * Eepro100 driver can sometimes report out of resources on reboot
       (Josue Emmanuel Amaro)

12. Probably Post 2.4

     * per super block write_super needs an async flag
     * per file_op rw_kiovec
     * rw sempahores on page faults (mmap_sem) (Currently protected by
       mmap_sem)
     * module remove race bugs (ipchains modules -- Rusty; won't fix for
       2.4)
     * NCR5380 isnt smp safe (Frank Davis --- belives the driver should
       be rewritten)
     * VM: physical->virtual reverse mapping, so we can do much better
       page aging with less CPU usage spikes
     * VM: better IO clustering for swap (and filesystem) IO
     * VM: move all the global VM variables, lists, etc. into the pgdat
       struct for better NUMA scalability
     * VM: (maybe) some QoS things, as far as they are major improvements
       with minor intrusion
     * VM: thrashing control, maybe process suspension with some forced
       swapping ?
     * VM: include Ben LaHaise's code, which moves readahead to the VMA
       level, this way we can do streaming swap IO, complete with
       drop_behind()
     * USB: spread out interrupt frames for devices that use the same
       interrupt period (interval)
     * USB: add USB 2.0 EHCI HCD
     _________________________________________________________________

Fixed

     * Incredibly slow loopback tcp bug (believed fixed about 2.3.48)
     * COMX series WAN now merged
     * VM needs rebalancing or we have a bad leak
     * SHM works chroot
     * SHM back compatibility
     * Intel i960 problems with I2O
     * Symbol clashes and other mess from _three_ copies of zlib!
     * PCI buffer overruns
     * Shared memory changes change the API breaking applications (eg
       gimp)
     * Finish softnet driver port over and cleanups
     * via rhine oopses under load ? S
     * SCSI generic driver crashes controllers (need to pass
       PCI_DIR_UNKNOWN..)
     * UMSDOS fixups resync (not quite done)
     * Make NTFS sort of work
     * Any user can crash FAT fs code with ftruncate
     * AFFS fixups
     * Directory race fix for UFS
     * Security holes in execve()
     * Lan Media WAN update for 2.3
     * Get the Emu10K merged
     * Paride seems to need fixes for the block changes yet
     * Kernel corrupts fs and gs in some situations (Ulrich has demo
       code)
     * 1.07 AMI MegaRAID
     * Merge 2.2.15 changes (Alan) x
     * Get RAID 0.90 in (Ingo)
     * S/390 Merge
     * NFS DoS fix (security)
     * Fix Space.c duplicate string/write to constants
     * Elevator and block handling queue change errors are all sorted
     * Make sure all drivers return 1 from their __setup functions (Done
       ?)
     * Enhanced disk statistics
     * Complete vfsmount merge (Al Viro)
     * Merge removed-buf-open directory stuff into VFS (Al Viro)
     * Problems with ip autoconfig according to Zaitcev
     * NFS causes dup kmem_create on reload (Trond)
     * vmalloc(GFP_DMA) is needed for DMA drivers (Ingo)
     * TLB flush should use highest priority (Ingo)
     * SMP affinity code creates multiple dirs with the same name (Ingo)
     * Set SMP affinity mask to actual cpu online mask (needed for some
       boards) (Ingo)
     * heavy swapping corrupts ptes (believed so)
     * pci_set_master forces a 64 latency on low latency setting
       devices.Some boards require all cards have latency <= 32
     * msync fails on NFS (probably fixed anyway)
     * Find out what has ruined disk I/O throughput. (mostly)
     * PIII FXSAVE/FXRESTORE support
     * The netdev name changing stuff broke GRE
     * put_user is broken for i386 machines (security) - sem stuff may be
       wrong too
     * BusLogic crashes when you cat /proc/scsi/BusLogic/0 (Robert de
       Vries)
     * Finish sorting out VM balancing (Rik Van Riel, Juan Quintela et
       al)
     * Fix eth= command line
     * 8139 + bridging fails
     * RtSig limit handling bug
     * Signals leak kernel memory (security) [FIX in ac tree]
     * TTY and N_HDLC layer called poll_wait twice per fd and corrupt
       memory
     * ATM layer calls poll_wait twice per fd and corrupts memory
     * Random calls poll_wait twice per fd and corrupts memory
     * PCI sound calls poll_wait twice per fd and corrupts memory
     * sbus audio calls poll_wait twice per fd and corrupts memory
     * IBM MCA driver breaks on Device_Inquiry at boot
     * SHM code corrupts memory (Russell)
     * Linux sends a 1K buffer with SCSI inquiries. The ANSI-SCSI limit
       is 255.
     * Linux uses TEST_UNIT_READY to chck for device presence on a
       PUN/LUN. The INQUIRY is the only valid test allowed by the spec.
     * truncate_inode_pages does unsafe page cache operations
     * Fix the ptrace code to be back compatible and add a new PTRACE
       call set for getting the PIII extra registers
     * EPIC100 fixes
     * Tlan and Epic100 crash under load
     * Fix hpfs_unlink (Al Viro)
     * exec loader permissions
     * Locking on getcwd
     * E820 memory setup causes crashes/corruption on some laptops[**VERY
       NASTY**] (fixed in test5)
     * Debian report that the gcc 2.95 possibly miscompiles fault.c or
       mm/remap.c (Perl script available from Arjan) (fixed in test2 or
       3)
     * Dcache threading (Al Viro)
     * Sockfs races (removing NULL ->i_sb stuf) (Al Viro)
     * Module remove race bug (done: anything with file_operations, fb
       stuff, procfs stuff - Al Viro)
     * DEFXX driver appears broken (reported fixed by Jeff Garzik)
     * Some FB drivers check the A000 area and find it busy then bomb out
       (checked and fixed, reported by Jeff Garzik)
     * Stick lock_kernel() calls around OSS driver with issues to hard to
       fix nicely for 2.4 itself (Alan, fixed)
     * Merge the current Compaq RAID driver into 2.4 (fixed, reported by
       Thomas Hiller)
     * mount crashes on Alpha platforms (fixed, reported by Thorsten
       Kranzkowski)
     * IDE fails on some VIA boards (eg the i-opener) (reported fixed by
       Konrad Stepien)
     * access_process_mm oops/lockup if task->mm changes (Manfred) [user
       can cause deliberately]
     * PCMCIA IRQ routing should now be fixed modulo ISA cards and bios
       doesn't tell us that an IRQ is ISA-only (Martin Mares)
     * TB Multisound driver hasnt been updated for new isa I/O totally.
       (reported fixed by John Coiner; see
       http://atv.ne.mediaone.net/linux-multisound)
     * yenta (PCMCIA) and pci_socket modules have mutual dependency
       (cardbus_register, yenta_operations) (test5, worked in test3)
       (reported fixed by Erik Mouw)
     * Keyboard/mouse problems (should be fixed?)
     * Floppy driver broken by VFS changes. Other drivers may be too
       (Stuff gets called after _close now - unload race possibly too;
       should be fixed in test6)
     * OSS module remove races (fixed by Christoph Hellwig)
     * Merge the 2.2 ServeRAID driver into 2.4 (Christoph Hellwig)
     * AHA27xx is broken (maybe 28xx too) (reported fixed by Doug
       Ledford)
     * Merge the network fixes (DaveM)
     * Finish 64bit vfs merges (lockf64 and friends missing -- willy?)
       (Andreas Jaeger reports that lockf64 has been added for Intel and
       Alpha; other architectures may not be done, but if not, they won't
       build :-)
     * Can't compile CONFIG_IBMTR and CONFIG_PCMCIA_IBMTR in kernel at
       once; kernel link failes (rasmus; fixed in a kludgy way by not
       allowing the combination by arjan)
     * Merge the RIO driver
     * Potential deadlock in EMU10K driver when running SMP (orre at
       nada.kth.se, Alan)
     * Symbol clashes from ppp and irda compression code (Arjan van de
       Ven)
     * Kernel build has race conditions when building modversions.h
       (Mikael Pettersson)
     * USB Pegasus driver explodes on disconnect (lots of printk and/or
       OOPS spewage to the console. David Ford) (reported fixed by Petko
       Manolov)
     * unsafe sleep_on in ibmcam and ov511 drivers (never was a problem,
       according to Mark McClelland)
     * netfilter doesn't compile correctly (test7-pre3, reported by Pau
       Aliagas, fixed by test7-pre6, rusty)
     * Innd data corruption, probably caused by bug truncation bug (Rik
       van Riel)
     * If all the ISO NLS's are modules, there can be an undefined ref to
       CONFIG_NLS_DEFAULT in inode.c (Dale Amon --- not a bug CONFIG_NLS
       is forced)
     * Fix sysinfo interface so it is binary compatible with 2.2.x (i.e.
       mem_unit=1), except when memory >= 4Gb (Erik Andersen)
     * Some people report 2.3.x serial problems (reported fixed by Shaya
       Potter)
     * Some Ultra-I sbus sparc64 systems fail to boot since 2.4.0-test3,
       may be due to specific memory configurations. (reported fixed by
       davem)
     * Fix, um, interesting races around dup2() and friends. (Al Viro)
     * complete the ext2 races fixes (truncate) (Al Viro)
     * USB pegasus driver doesn't work since 2.4.0test5 (David Ford)
     * Splitting a posix lock causes an infinite loop (Stephen Rothwell,
       according to Rusty)
     * Oops in dquot_transfer (David Ford, Martin Diehl) (Jan Kara has a
       potential patch from 2.2, submitted to Linus by Martin Diehl,
       fixed in test9)
     * SHM segments not always being detached and destroyed right ?
       (problem reported by Lincoln Dale (was combination of XFree86 and
       kernel bug, fixed))
     * Mount of new fs over existing mointpoint should return an error
       unless forced (Andrew McNabb, Alan Cox) (Andries Brouwer has
       posted a patch)
     * Boot hangs on a range of Dell docking stations (Latitude, reported
       fixed in 2.4.0-pre9)
          + Almost certainly related: PCI code doesn't see devices behind
            DECchip 21150 PCI bridges (used in Dell Latitude). Reported
            by Simon Trimmer. (Patch from Martin Mares exists but it
            disables cardbus devices, according to Tigran.)
          + Derek Fawcus at Cisco reports similar problems with Toshiba
            Tecra 8000 attached to the DeskStation V+ docking station.
            (once again, caused by bridge returning 0 when reading the
            I/O base/limit and Memory base/limit registers which confuses
            the new PCI resource code).
     * drivers/sound/cs46xx.c has compile errors test7 and test8 (C
       Sanjayan Rosenmund, reported fixed by Hayden James)
     * Network block device seems broken by block device changes (Jeffrey
       C. Becker reports no problems)
     * cdrecord doesn't work (produces CD-ROM coasters) w/o any errors
       reported, works under 2.2 (Damon LoCascio, reported as fixed by
       Robert M. Love)
     * ACPI hangs on boot for some systems (Are there any cases left?
       Reported as fixed by Simon Richter)
     * arcnet/com20020-isa.c doesn't compile, as of 2.4.0-test8. Dan
       Aloni has a fix, reported fixed)
     * 2.4.0-test8 has a BUG at ll_rw_blk:711. (Johnny Accot, Steffen
       Luitz) (Al Viro has a patch, reported fixed by Udo A. Steinberg)
     * Writing to tapes > 2.4G causes tar to fail with EIO (using
       2.4.0-test7-pre5; it works under 2.4.0-test1-ac18 --- Tigran
       Aivazian, reported fixed)
     * 2.4.0-test2 breaks the behaviour of the ether=0,0,eth1 boot
       parameter (dwguest, reported as fixed.)
     * IBM Thinkpad 390 won't boot since 2.3.11 (Decklin Foster; NOT A
       BUG; caused by misconfigured CPU model)
     * PPC-specific: won't boot on 601 CPU's (powermac) (Andreas Tobler;
       Paul Mackerras has fix in PPC tree)
     * Finish I2O merge (Intel/Alan, reported as fixed as it's going to
       be)
     * Non-atomic page-map operations can cause loss of dirty bit on
       pages (sct, alan, Ben LaHaise fixed)
     * NFS V3 lockd causes kernel oops (Trond Myklebust, reported fixed)
     * VM: Out of Memory handling {CRITICAL} (added in test10)
     * TLAN nic appears to be adding a timer twice (2.4.0test8pre6, Arjan
       ve de Ven) (Fixed, but patch not sent to Linus yet -- Torben
       Mathiasen)
     * Loading the qlogicfc driver in 2.4.0-test8 causes the kernel to
       loop forver reporting SCSI disks that aren't present (Paul
       Hubbard, Torben Mathiasen has a potential patch, sent to Linus,
       need to very with Paul)
     * Fix the minixfs races (Al Viro)
     * SIT tunneling (ipv6 in ipv4) is broken (Gerhard Mack; Davem has a
       fix) (fixed in test10-pre2)
     * USB: hid joystick handling (patch from Vajtech, 2.4.0.9.4)
     * USB: cpia_usb module doesn't handle "no bandwidth" returns (OOPS)
       (2.4.0.9.4)
     * USB: microtek memory handling (patch from Oliver Neukum)
       (2.4.0.9.4)
     * USB: usb-uhci not use set PCI Latency Timer register to 0
     * USB: printer driver aborts on out-of-paper or off-line conditions
       instead of retrying until the condition is fixed
     * USB: usb-uhci SMP spinlock/bad pointer crash
     * USB: cpia camera driver with OHCI HCD locks up or fails
     * USB: pegasus (ethernet) driver crashes often
     * USB: Fix differences in UHCI and OHCI HCD behaviors/semantics:
          + Only uhci.c does EARLY_COMPLETE (drop EARLY_COMPLETE ?)
          + USB: Fix the OOPS in usb-storage from the error-recovery
            handler. {CRITICAL} (reported by Matthew Dharm, fixed as of
            test9)
          + USB: booting with USB compiled into kernel causes a lot of
            syslog entries as the root hubs are probed by all drivers
            (this is especially obnoxious as the usb-serial drivers start
            up) (Fixed in test9, according to Greg KH)
          + USB: fix setting urb->dev in plusb, wacom, mdc800) (Greg KH)
            {CRITICAL} (fixed in test10-pre1)
          + USB: fix usb-uhci setting urb->dev = NULL at correct places
            only {CRITICAL} (fixed in test10-pre1)
          + USB: pegasus driver version 0.4.12: update to work with HCD
            changes; fix module use counting & dev refcounting; fix to
            work with dhcpd (Petko) {CRITICAL} (fixed in test10-pre1)
          + USB: pegasus driver locks up on slow oHCI machine; sometimes
            cannot be rmmod-ed (Cyrille Chepelov) (was uhci problem
            fixedin test10-pre1 according to Petko Manolov)
          + USB: oops on boot with 2.4.0-test9, usb-uhci, pegasus (in
            process_transfer) (frank@unternet.org) (was uhci problem
            fixedin test10-pre1 according to Petko Manolov)
          + Sys_revoke() (CRITICAL, revoke or some subset of it is needed
            to fix some security issues) (alan, linus, worked around for
            now)
          + USB: hotplug (PNP) and module autoloader support (currently
            being tested)
          + USB: OHCI root-hub-timer does not restart on resume
            {CRITICAL} (Paul Mackerras)
          + USB: add bandwidth allocation support to usb-uhci HCD
          + USB: usb-ohci needs to null urb->dev to avoid various
            reboots/hangs/oopses {CRITICAL} (David Brownell)
          + USB: speed up device enumeration (hub driver has large delays
            in it)
          + USB: OOPS when unplugging mouse from external hub (Unable to
            handle kernel paging request at virtual address 00080004; in
            usb_disconnect) (2.4.0-test9-pre7)
          + USB: With uhci HCD, switching from X to a text console and
            back to X, USB mouse is dead. (2.4.0-test9-pre7)
          + USB: plusb oops, segfaults, performance.
          + USB: usb-uhci, microtek scanner driver on 2.4.0-test7 & SANE
            1.0.3 OOPSes; usb-ohci works.
            >[usb-uhci]uhci_cleanup_unlink+4c/140<
          + USB: 2.4.0-test9-pre9: Novatek Ortek USB kbd not working.
          + USB: 2.4.0-test9-pre9: unresolved USB symbols when only
            usbcore is in-kernel.
          + USB: 2.4.0-test10: unresolved symbol 'this_module' when
            compiling only usbcore in kernel (in inode.c).
          + USB: 2.4.0-test9: USB + SMP gives lots of USB device timeout
            errors. OK without SMP. Tyan tiger 133 (1854d, i believe)
            mobo (via apollo pro 133a chipset). Redhat 6.2 and 7.0
            (thought it might be gcc 2.96, but it seems to happen with
            both). Either UHCI HCD. Or Asus p2b-ds mobo with the intel bx
            chipset with same results.
          + USB: test9, test10-pre1, and 2.2.18-pre15: OOPS with USB
            mouse. >>EIP; c0121491 >kmem_cache_free+14d/174< >=====
            Trace; d00387a6 >[usb-uhci]uhci_unlink_urb_sync+122/150<
          + USB: many unexplained "usb_control/bulk_msg: timeout"
            messages on several different USB devices.
          + USB: 3Com USB ISDN TA not working with test9-pre3 or
            test10-pre4. Worked with test8.

Probably Hardware Bugs

     * Data corruption on IDE disks (Generic PCI DMA and SiS support
       Steven Walter) (sounds like PCChips #M599LMR motherboard doesn't
       disable UDMA when a non-UDMA cable is used. If you disable UDMA in
       the BIOS, then there is no problem. hardware bug?)
     * AHA29xx driver appears to stomp other cards (may be BIOS; probably
       motherboard has assigned to small of a range to a card, so that
       it's overlapping with some other card -- Doug Ledford)
     * USB hangs on APM suspend on some machines (fixed more most; Alan
       has one still that fails but the BIOS has 'issues')
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Alan Cox]

>      * Palmax PD1100 hangs during boot since 2.4.0-test9 (Alan Cox)

Fixed in pre10.

>      * AIC7xxx doesnt work non PCI ? (Doug says OK, new version due
>        anyway)

This is now in Justin Gibbs hand but will take time to move on. Doug confirmed
his current code is now merged too.

>      * Check all devices use resources properly (Everyone now has to use
>        request_region and check the return since we no longer single
>        thread driver inits in all module cases. Also memory regions are
>        now requestable and a lot of old drivers dont know this yet. --
>        Alan Cox)

Folks have done most of the common drivers. So its not really a show stopper
now just a 'clean up'

>      * Issue with notifiers that try to deregister themselves? (lnz;
>        notifier locking change by Garzik should backed out, according to
>        Jeff)

and according to Alan

>      * SCSI CD-ROM doesn't work on filesystems with < 2kb block size
>        (Jens Axboe will fix)

SCSI M/O has the same problem. 2.2 can mount MSDOS fs on M/O 2.4test 10 cant.

>      * FAT filesystem doesn't support 2kb sector sizes (did under 2.2.16,
>        doesn't under 2.4.0test7. Kazu Makashima, alan)

This is the same as the SCSI cd rom issue. You can either do reblocking in the
fat layer and other fs's needing it or do it in the scsi code.

>      * Spin doing ioctls on a down netdeice as it unloads == BOOM
>        (prumpf, Alan Cox) Possible other net driver SMP issues (andi
>        kleen)

Turns out to be safe according to Jeff and ANK

> 10. To Do But Non Showstopper
>      * PCMCIA/Cardbus hangs (Basically unusable - Hinds pcmcia code is
>        reliable)
>           + PCMCIA crashes on unloading pci_socket

The pci_socket crash is fixed it seems

>      * Some AWE cards are not being found by ISAPnP ??

You have this one higher up as problems with some SB AWE cards

Alan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Philipp Rumpf]

> 
> 3. Security
> 
>      * Fix module remove race bug (still to be done: TTY, ldisc, I2C,
>        video_device - Al Viro) (Rogier Wolff will handle ATM)

TBD: sysctl, kernel_thread

* drivers/input/mousedev.c dereferences userspace pointers directly.  We
should make this fail for a bit to catch those bugs
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Andi Kleen]

> >      * Spin doing ioctls on a down netdeice as it unloads == BOOM
> >        (prumpf, Alan Cox) Possible other net driver SMP issues (andi
> >        kleen)
> 
> Turns out to be safe according to Jeff and ANK

It is not safe, just not worse than 2.2.

-Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

loop device hangs

[Christian van Enckevort]

> 10. To Do But Non Showstopper
> 
>      * Loop device can still hang (William Stearns has script that will
>        hang 2.4.0-test7, Peter Enderborg has a short sequence that will
>        hang 2.4.0test9)
I think I have the same problem with 2.4.0-test10 (with rieserfs-patch). 
With small images (initrd-ramdisk) I have no problem, but when I prepare an
ext2fs-image for a backup on CD, the system will invariably hang. For me
this is a showstopper bug. I tried to further investigate this bug. I found
it can also be triggered by running bonnie on an ext2fs-image
(filesize=200Mb). When it hung, the EIP pointed to somewhere in
default_idle. Does anybody have some hints for finding some more useful
information about this bug?

Christian van Enckevort
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[kuznet]

Hello!

> It is not safe, just not worse than 2.2.

Andi...... 8)

That issue, which was meant here, it is 100% safe.

I start to suspect you did not look into this code since 2.2.
I acn assume that nothing has changed in ISDN,
in other drivers big changes happened. 8)

Alexey
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[David Ford]

[-- Attachment #1: Type: text/plain, Size: 632 bytes --]

Alan Cox wrote:

> > 10. To Do But Non Showstopper
> >      * PCMCIA/Cardbus hangs (Basically unusable - Hinds pcmcia code is
> >        reliable)
> >           + PCMCIA crashes on unloading pci_socket
>
> The pci_socket crash is fixed it seems

Not unless it was fixed in test10 release.  I have a PC LinkSys dual 10/100 and
56K card that will kill the machine if you physically pull it out no matter what
cardctl/module steps are taken.

It uses the ne2k and serial drivers.

-d

--
"The difference between 'involvement' and 'commitment' is like an
eggs-and-ham breakfast: the chicken was 'involved' - the pig was
'committed'."



[-- Attachment #2: Card for David Ford --]
[-- Type: text/x-vcard, Size: 239 bytes --]

begin:vcard 
n:Ford;David
x-mozilla-html:TRUE
org:<img src="http://www.kalifornia.com/images/paradise.jpg">
adr:;;;;;;
version:2.1
email;internet:david@kalifornia.com
title:Blue Labs Developer
x-mozilla-cpt:;-12480
fn:David Ford
end:vcard

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Jeff Garzik]

David Ford wrote:
> 
> Alan Cox wrote:
> 
> > > 10. To Do But Non Showstopper
> > >      * PCMCIA/Cardbus hangs (Basically unusable - Hinds pcmcia code is
> > >        reliable)
> > >           + PCMCIA crashes on unloading pci_socket
> >
> > The pci_socket crash is fixed it seems
> 
> Not unless it was fixed in test10 release.  I have a PC LinkSys dual 10/100 and
> 56K card that will kill the machine if you physically pull it out no matter what
> cardctl/module steps are taken.
> 
> It uses the ne2k and serial drivers.

Part of that might be that serial doesn't support hotplug without
patching.

-- 
Jeff Garzik             | Dinner is ready when
Building 1024           | the smoke alarm goes off.
MandrakeSoft            |	-/usr/games/fortune
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Jeff Garzik]

Alan Cox wrote:
> >      * Check all devices use resources properly (Everyone now has to use
> >        request_region and check the return since we no longer single
> >        thread driver inits in all module cases. Also memory regions are
> >        now requestable and a lot of old drivers dont know this yet. --
> >        Alan Cox)
> 
> Folks have done most of the common drivers. So its not really a show stopper
> now just a 'clean up'

s/check_region/request_region/ is moving along, but there is still a
ways to go.  I agree with "folks have done most of the common drivers"

I have seen very few additions of request_mem_region, though.


> >      * Issue with notifiers that try to deregister themselves? (lnz;
> >        notifier locking change by Garzik should backed out, according to
> >        Jeff)
> 
> and according to Alan

Fixed for a couple versions now.


> >      * Spin doing ioctls on a down netdeice as it unloads == BOOM
> >        (prumpf, Alan Cox) Possible other net driver SMP issues (andi
> >        kleen)
> 
> Turns out to be safe according to Jeff and ANK

To avoid Andi confusing the issue :) ...

1) The first item listed does not exist

2) the second item listed exists -- many net drivers are not SMP.  They
are most of the way there, but there are still small races which exist
in some drivers.

	Jeff



-- 
Jeff Garzik             | Dinner is ready when
Building 1024           | the smoke alarm goes off.
MandrakeSoft            |	-/usr/games/fortune
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[David Ford]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

The odd part is that it used to work "way back when".  Was this just a fluke?

-d

Jeff Garzik wrote:

> > Not unless it was fixed in test10 release.  I have a PC LinkSys dual 10/100 and
> > 56K card that will kill the machine if you physically pull it out no matter what
> > cardctl/module steps are taken.
> >
> > It uses the ne2k and serial drivers.
>
> Part of that might be that serial doesn't support hotplug without
> patching.

--
"The difference between 'involvement' and 'commitment' is like an
eggs-and-ham breakfast: the chicken was 'involved' - the pig was
'committed'."



[-- Attachment #2: Card for David Ford --]
[-- Type: text/x-vcard, Size: 239 bytes --]

begin:vcard 
n:Ford;David
x-mozilla-html:TRUE
org:<img src="http://www.kalifornia.com/images/paradise.jpg">
adr:;;;;;;
version:2.1
email;internet:david@kalifornia.com
title:Blue Labs Developer
x-mozilla-cpt:;-12480
fn:David Ford
end:vcard

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Jeff Garzik]

tytso@mit.edu wrote:
> 4. Boot Time Failures
> 
>      * Crashes on boot on some Compaqs ? (may be fixed)

compaq laptops?  desktops?  or alphas?


>      * Various Alpha's don't boot under 2.4.0-test9 (PCI resource
>        allocation problem? Michal Jaegermann; Richard Henderson may have
>        an idea what's failing.)

Elaboration:  PCI-PCI bridges are not configured correctly

> 5. Compile errors

I doubt you need this category :)


> 6. In Progress
>      * Fix all remaining PCI code to use pci_enable_device (mostly done)

Most drivers are done, and all of the important drivers are done
(IMHO).  Maybe we could move this to a cleanup category?  Its definitely
not a showstopper..


>      * DMFE is not SMP safe (Frank Davis patch exists, but hasn't gotten
>        much commens yet)

update:  frank davis patch is poor.

DMFE accesses multiple hardware registers for a single operation, and
requires SMP locking to synchronize between all that code.


>      * Audit all char and block drivers to ensure they are safe with the
>        2.3 locking - a lot of them are not especially on the
>        read()/write() path. (Frank Davis --- moving slowly; if someone
>        wants to help, contact Frank)

Haven't heard any update on this in a long while...


>      * Fixing autofs4 to deal with VFS changes (Jeremy Fitzhardinge)

I thought this was complete a long time ago?


> 8. Fix Exists But Isnt Merged

>      * Many network device drivers don't call MOD_INC_USE_COUNT in
>        dev->open. (Paul Gortmaker has patches)

There exists a patch which makes MOD_xxx in net drivers obsolete.  I'm
hoping that one will get applied...


>      * mtrr.c is broken for machines with >= 4GB of memory (David Wragg
>        has a fix)

His patch looks ok to me, too....   Does somebody want to submit this
patch to Linus?  I haven't seen the maintainer (Richard Gooch) speak up
on this issue at all.


>      * Issue with notifiers that try to deregister themselves? (lnz;
>        notifier locking change by Garzik should backed out, according to
>        Jeff)

Done.


>      * The new hot plug PCI interface does not provide a method for
>        passing the correct device name to cardmgr (David Hinds, alan)

Move to "in progress"...


>      * 2.4.0-test8 pcmcia is unusable in fall forms (kernel, mixed, or
>        dhinds code) (David Ford)

"fall forms"?

David clearly has problems w/ pcmcia, but it is not at all as broken as
he makes it out to be:  all my cardbus laptops boot and work.


>      * Spin doing ioctls on a down netdeice as it unloads == BOOM
>        (prumpf, Alan Cox)

not an issue.


>	Possible other net driver SMP issues (andi
>        kleen)

No showstoppers AFAICS, but small races do exist.


>      * PCMCIA/Cardbus hangs (Basically unusable - Hinds pcmcia code is
>        reliable)

Again "whatever".  The CardBus code is definitely usable.  It is not
mature, but saying it is "basically unusable" is wildly inaccurate.


>      * RTL 8139 cards sometimes stop responding. Both drivers don't
>        handle this quite good enough yet. (reported by Rogier Wolff,
>        tentatively reported as fixed by David Ford; reports from Frank
>        Jacobberger and Shane Shrybman indicate that it doesn't appear to
>        be fixed in test9)

I'm hoping this is fixed in test10, but haven't gotten any reports back
yet...


>      * kiobuf seperate lock functions/bounce/page_address fixes

Do Stephen Tweedie's recently posted kiobuf patches fix this issue?


>      * Potential races in file locking code (Christian Ehrhardt)
>           + locks_verify_area checks the wrong range if O_APPEND is set
>             and the current file position is not at the end of the file.
>           + dito if the file position changes between the call to
>             locks_verify_area and the actual read/write (requires a
>             shared file pointer, an attacker can use this to circumvent
>             virtually any mandatory lock).
>           + active writes should prevent anyone from getting mandatory
>             locks for the area beeing written.
>           + active reads should prevent anyone from getting mandatory
>             write locks for the area beeing read.

a fix patch for file locks (related to nfs, but still it appears to fix
some general issues)  was posted this week:  
http://boudicca.tux.org/hypermail/linux-kernel/this-week/0404.html


>      * Eepro100 driver can sometimes report out of resources on reboot
>        (Josue Emmanuel Amaro)

More than just on reboot.

-- 
Jeff Garzik             | Dinner is ready when
Building 1024           | the smoke alarm goes off.
MandrakeSoft            |	-/usr/games/fortune
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Alan Cox]

> > Not unless it was fixed in test10 release.  I have a PC LinkSys dual 10/100 and
> > 56K card that will kill the machine if you physically pull it out no matter what
> > cardctl/module steps are taken.
> > 
> > It uses the ne2k and serial drivers.
> 
> Part of that might be that serial doesn't support hotplug without
> patching.

Linksys rings a bell with an outstandng 2.2 lockup on pcmcia. I think this might
be a driver bug ?

> 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[James Simmons]

>      * VGA Console can cause SMP deadlock when doing printk {CRITICAL}
>        (Keith Owens)

Still not fixed :-( Here is the patch again. Keith give it a try and tell
me if it solves your problems.

--- vgacon.c.orig	Tue Oct 24 18:45:58 2000
+++ vgacon.c	Tue Oct 24 19:08:51 2000
@@ -46,11 +46,13 @@
 #include <linux/malloc.h>
 #include <linux/vt_kern.h>
 #include <linux/selection.h>
+#include <linux/spinlock.h>
 #include <linux/ioport.h>
 #include <linux/init.h>
 
 #include <asm/io.h>
 
+static spinlock_t vga_lock = SPIN_LOCK_UNLOCKED;
 
 #define BLANK 0x0020
 
@@ -152,8 +154,7 @@
 	 * ddprintk might set the console position from interrupt
 	 * handlers, thus the write has to be IRQ-atomic.
 	 */
-	save_flags(flags);
-	cli();
+	spin_lock_irqsave(&vga_lock, flags);	
 
 #ifndef SLOW_VGA
 	v1 = reg + (val & 0xff00);
@@ -166,7 +167,7 @@
 	outb_p(reg+1, vga_video_port_reg);
 	outb_p(val & 0xff, vga_video_port_val);
 #endif
-	restore_flags(flags);
+	spin_unlock_irqrestore(&vga_lock, flags);
 }
 
 static const char __init *vgacon_startup(void)
@@ -415,7 +416,7 @@
 	if ((from == lastfrom) && (to == lastto)) return;
 	lastfrom = from; lastto = to;
 
-	save_flags(flags); cli();
+	spin_lock_irqsave(&vga_lock, flags);
 	outb_p(0x0a, vga_video_port_reg);		/* Cursor start */
 	curs = inb_p(vga_video_port_val);
 	outb_p(0x0b, vga_video_port_reg);		/* Cursor end */
@@ -428,7 +429,7 @@
 	outb_p(curs, vga_video_port_val);
 	outb_p(0x0b, vga_video_port_reg);		/* Cursor end */
 	outb_p(cure, vga_video_port_val);
-	restore_flags(flags);
+	spin_unlock_irqrestore(&vga_lock, flags);
 }
 
 static void vgacon_cursor(struct vc_data *c, int mode)
@@ -533,11 +534,11 @@
 {
 	/* save original values of VGA controller registers */
 	if(!vga_vesa_blanked) {
-		cli();
+		spin_lock_irq(&vga_lock);
 		vga_state.SeqCtrlIndex = inb_p(seq_port_reg);
 		vga_state.CrtCtrlIndex = inb_p(vga_video_port_reg);
 		vga_state.CrtMiscIO = inb_p(video_misc_rd);
-		sti();
+		spin_unlock_irq(&vga_lock);
 
 		outb_p(0x00,vga_video_port_reg);	/* HorizontalTotal */
 		vga_state.HorizontalTotal = inb_p(vga_video_port_val);
@@ -561,7 +562,7 @@
 
 	/* assure that video is enabled */
 	/* "0x20" is VIDEO_ENABLE_bit in register 01 of sequencer */
-	cli();
+	spin_lock_irq(&vga_lock);
 	outb_p(0x01,seq_port_reg);
 	outb_p(vga_state.ClockingMode | 0x20,seq_port_val);
 
@@ -598,13 +599,13 @@
 	/* restore both index registers */
 	outb_p(vga_state.SeqCtrlIndex,seq_port_reg);
 	outb_p(vga_state.CrtCtrlIndex,vga_video_port_reg);
-	sti();
+	spin_unlock_irq(&vga_lock);
 }
 
 static void vga_vesa_unblank(void)
 {
 	/* restore original values of VGA controller registers */
-	cli();
+	spin_lock_irq(&vga_lock);
 	outb_p(vga_state.CrtMiscIO,video_misc_wr);
 
 	outb_p(0x00,vga_video_port_reg);		/* HorizontalTotal */
@@ -629,7 +630,7 @@
 	/* restore index/control registers */
 	outb_p(vga_state.SeqCtrlIndex,seq_port_reg);
 	outb_p(vga_state.CrtCtrlIndex,vga_video_port_reg);
-	sti();
+	spin_unlock_irq(&vga_lock);
 }
 
 static void vga_pal_blank(void)
@@ -750,7 +751,7 @@
 		charmap += 4*cmapsz;
 #endif
 
-	cli();
+	spin_lock_irq(&vga_lock);
 	outb_p( 0x00, seq_port_reg );   /* First, the sequencer */
 	outb_p( 0x01, seq_port_val );   /* Synchronous reset */
 	outb_p( 0x02, seq_port_reg );
@@ -766,7 +767,7 @@
 	outb_p( 0x00, gr_port_val );    /* disable odd-even addressing */
 	outb_p( 0x06, gr_port_reg );
 	outb_p( 0x00, gr_port_val );    /* map start at A000:0000 */
-	sti();
+	spin_unlock_irq(&vga_lock);
 	
 	if (arg) {
 		if (set)
@@ -793,7 +794,7 @@
 		}
 	}
 	
-	cli();
+	spin_lock_irq(&vga_lock);
 	outb_p( 0x00, seq_port_reg );   /* First, the sequencer */
 	outb_p( 0x01, seq_port_val );   /* Synchronous reset */
 	outb_p( 0x02, seq_port_reg );
@@ -833,8 +834,7 @@
 		inb_p( video_port_status );
 		outb_p ( 0x20, attrib_port );
 	}
-	sti();
-
+	spin_unlock_irq(&vga_lock);
 	return 0;
 }
 
@@ -865,12 +865,12 @@
 	   registers; they are write-only on EGA, but it appears that they
 	   are all don't care bits on EGA, so I guess it doesn't matter. */
 
-	cli();
+	spin_lock_irq(&vga_lock);
 	outb_p( 0x07, vga_video_port_reg );		/* CRTC overflow register */
 	ovr = inb_p(vga_video_port_val);
 	outb_p( 0x09, vga_video_port_reg );		/* Font size register */
 	fsr = inb_p(vga_video_port_val);
-	sti();
+	spin_lock_irq(&vga_lock);
 
 	vde = maxscan & 0xff;			/* Vertical display end reg */
 	ovr = (ovr & 0xbd) +			/* Overflow register */
@@ -878,14 +878,14 @@
 	      ((maxscan & 0x200) >> 3);
 	fsr = (fsr & 0xe0) + (fontheight-1);    /*  Font size register */
 
-	cli();
+	spin_lock_irq(&vga_lock);
 	outb_p( 0x07, vga_video_port_reg );		/* CRTC overflow register */
 	outb_p( ovr, vga_video_port_val );
 	outb_p( 0x09, vga_video_port_reg );		/* Font size */
 	outb_p( fsr, vga_video_port_val );
 	outb_p( 0x12, vga_video_port_reg );		/* Vertical display limit */
 	outb_p( vde, vga_video_port_val );
-	sti();
+	spin_unlock_irq(&vga_lock);	
 
 	vc_resize_all(rows, 0);			/* Adjust console size */
 	return 0;



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Jeff Garzik]

Alan Cox wrote:
> 
> > > Not unless it was fixed in test10 release.  I have a PC LinkSys dual 10/100 and
> > > 56K card that will kill the machine if you physically pull it out no matter what
> > > cardctl/module steps are taken.
> > >
> > > It uses the ne2k and serial drivers.
> >
> > Part of that might be that serial doesn't support hotplug without
> > patching.
> 
> Linksys rings a bell with an outstandng 2.2 lockup on pcmcia. I think this might
> be a driver bug ?

On 2.2.x, possibly.

On 2.4.x:  1) insert CardBus serial or modem card, that reports itself
as PCI_CLASS_COMMUNICATION_SERIAL.  2) modprobe serial, it sees and
registers the PCI serial port.  3) eject card, which serial.c doesn't
presently notice.  ...

	Jeff


-- 
Jeff Garzik             | Dinner is ready when
Building 1024           | the smoke alarm goes off.
MandrakeSoft            |	-/usr/games/fortune
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Jeff Garzik]

David Ford wrote:
> The odd part is that it used to work "way back when".  Was this just a fluke?

That may have been back in the legacy days.  Ejecting ne2k should be ok
as long as you are using ne2k-pci or pcnet_cs.  Eject serial looks like
bad news unless you are using serial_cs (which is impossible at the
moment, AFAIK).

-- 
Jeff Garzik             | Dinner is ready when
Building 1024           | the smoke alarm goes off.
MandrakeSoft            |	-/usr/games/fortune
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Keith Owens]

On Fri, 3 Nov 2000 17:10:52 -0800 (PST), 
James Simmons <jsimmons@suse.com> wrote:
>
>>      * VGA Console can cause SMP deadlock when doing printk {CRITICAL}
>>        (Keith Owens)
>
>Still not fixed :-( Here is the patch again. Keith give it a try and tell
>me if it solves your problems.

The patch looks OK to me.  I worked around my original problem (screen
deadlock in kdb) by skipping the cli()/restore_flags() when kdb is
active, kdb guarantees that only one cpu at a time is doing any work.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[David Ford]

[-- Attachment #1: Type: text/plain, Size: 1934 bytes --]

Jeff Garzik wrote:

> >      * 2.4.0-test8 pcmcia is unusable in fall forms (kernel, mixed, or
> >        dhinds code) (David Ford)
>
> "fall forms"?
>
> David clearly has problems w/ pcmcia, but it is not at all as broken as
> he makes it out to be:  all my cardbus laptops boot and work.
>
>
> >      * PCMCIA/Cardbus hangs (Basically unusable - Hinds pcmcia code is
> >        reliable)
>
> Again "whatever".  The CardBus code is definitely usable.  It is not
> mature, but saying it is "basically unusable" is wildly inaccurate.

The qualifiers I reported are not included above so don't take it to mean
wide ranging.

I reported pcmcia in all forms was broken for test8 on -my hardware-.

Other kernels such as test10-prex that I'm on now are workable with dhinds
pcmcia.  I sent you all the requested information you asked for in several
forms.  The kernel's idea of the the sockets just doesn't work...again, on
-my hardware-.

It doesn't matter what voodoo you practice, all of the kernels in the last
year have been unable to drive -my hardware- in any sort of stable fashion.
Recent kernels just can't figure out IRQs for the sockets.

David's package works in all situations except for the combined ne2k/modem
that I reported earlier and the ray_cs in similar fashion.

For the second report, when the kernel did figure out IRQs for the sockets,
plugging in a card sometimes killed all software interrupts.  I.e. hardware
responded such as caps, screen blank/active etc, but no mouse or key events
made it past the kernel.  Unplugging a card or putting it in socket 0
normally caused a plethura of unending OOPSes or another hang on the
insertion.

Ted, please put my qualifier back in, "On this NEC Versa LX laptop", I don't
want my reports taken out of context. :)

-d


--
"The difference between 'involvement' and 'commitment' is like an
eggs-and-ham breakfast: the chicken was 'involved' - the pig was
'committed'."



[-- Attachment #2: Card for David Ford --]
[-- Type: text/x-vcard, Size: 239 bytes --]

begin:vcard 
n:Ford;David
x-mozilla-html:TRUE
org:<img src="http://www.kalifornia.com/images/paradise.jpg">
adr:;;;;;;
version:2.1
email;internet:david@kalifornia.com
title:Blue Labs Developer
x-mozilla-cpt:;-12480
fn:David Ford
end:vcard

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[David Ford]

[-- Attachment #1: Type: text/plain, Size: 725 bytes --]

Alan Cox wrote:

> > > Not unless it was fixed in test10 release.  I have a PC LinkSys dual 10/100 and
> > > 56K card that will kill the machine if you physically pull it out no matter what
> Linksys rings a bell with an outstandng 2.2 lockup on pcmcia. I think this might
> be a driver bug ?

I suspect it might be, I also think it may be getting the wrong voltage or it's poorly
designed hardware.  It gets hot enough to make a blister if you don't handle it
carefully.

It works fine all the while installed, goes for days on end, even tho it gets
incredibly hot. :)

-d

--
"The difference between 'involvement' and 'commitment' is like an
eggs-and-ham breakfast: the chicken was 'involved' - the pig was
'committed'."



[-- Attachment #2: Card for David Ford --]
[-- Type: text/x-vcard, Size: 239 bytes --]

begin:vcard 
n:Ford;David
x-mozilla-html:TRUE
org:<img src="http://www.kalifornia.com/images/paradise.jpg">
adr:;;;;;;
version:2.1
email;internet:david@kalifornia.com
title:Blue Labs Developer
x-mozilla-cpt:;-12480
fn:David Ford
end:vcard

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Keith Owens]

On Fri, 3 Nov 2000 10:09:31 -0500, 
tytso@mit.edu wrote:
>9. To Do
>     * DRM cannot use AGP support module when CONFIG_MODVERSIONS is
>       defined (issue with get_module_symbol caused fix proposed by John
>       Levon to be rejected)

Move this to "in progress" and add MTD code breaks with
CONFIG_MODVERSIONS, for the same reason.  I wrote a patch to replace
get_module_symbol a week ago and sent it to the DRM/AGP/MTD people for
testing - no response yet.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Stephen C. Tweedie]

Hi,

On Fri, Nov 03, 2000 at 05:20:53PM -0500, Jeff Garzik wrote:
> 
> >      * kiobuf seperate lock functions/bounce/page_address fixes
> 
> Do Stephen Tweedie's recently posted kiobuf patches fix this issue?

Hopefully, but not 100%: there is still a race window on anonymous
pages which needs to be fixed elsewhere in the VM.  The window for
mmaped files is closed in those patches, though.

--Stephen
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Russell King]

Keith Owens writes:
> Move this to "in progress" and add MTD code breaks with
> CONFIG_MODVERSIONS, for the same reason.  I wrote a patch to replace
> get_module_symbol a week ago and sent it to the DRM/AGP/MTD people for
> testing - no response yet.

<aol>
I have a fix likewise for the MTD code, so it builds without CONFIG_MODULES
</aol>
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |         Russell King        rmk@arm.linux.org.uk      --- ---
  | | | | http://www.arm.linux.org.uk/personal/aboutme.html   /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[David Woodhouse]

On Sat, 4 Nov 2000, Keith Owens wrote:

> Move this to "in progress" and add MTD code breaks with
> CONFIG_MODVERSIONS, for the same reason.  I wrote a patch to replace
> get_module_symbol a week ago and sent it to the DRM/AGP/MTD people for
> testing - no response yet.

Sorry for the delay. I don't actually have any appropriate hardware any
more that doesn't now have a JFFS root filesystem, making it difficult to
test the modular code :)

Your patch looks like it'll work. Although I don't really see any
advantage over {get,put}_module_symbol() in this case, it does look like
it can be used to finally provide module persistent storage, which will be
useful.

However, the easy fix for the MTD code is to use EXPORT_SYMBOL_NOVERS()
for the offending symbols. It's the most appropriate for putting into 2.4.

If the inter_module_{put,get} change is really going into 2.4 at this
late stage, then I'll merge your patch into my CVS tree and look at
updating the MTD code in the 2.4 tree to a more recent version. I was
intending to leave that for later, though.

Also - if it goes into 2.4, please make sure it goes into 2.2 as well.
get_module_symbol() is already broken there because it doesn't increase
the module's use count, and it'll prevent an ugly mess of ifdefs.

-- 
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Keith Owens]

On Sun, 5 Nov 2000 23:15:27 +0000 (GMT), 
David Woodhouse <dwmw2@infradead.org> wrote:
>Your patch looks like it'll work. Although I don't really see any
>advantage over {get,put}_module_symbol() in this case, it does look like
>it can be used to finally provide module persistent storage, which will be
>useful.

Cleanliness.  All other module data flows go via explicit symbols which
are fixed up at link time or via registration functions.  Having a few
sources that use a third mechanism which forces people to use
EXPORT_SYMBOL_NOVERS() to make it work is messy and redundant.

I'm not sure why you think this can be used for module persistent
storage.  If a module calls inter_module_register() on load, it should
call inter_module_unregister() on unload.  All the registered data
points into the loaded module, remove the module and the storage
disappears as well.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[David Woodhouse]

On Mon, 6 Nov 2000, Keith Owens wrote:

> I'm not sure why you think this can be used for module persistent
> storage.  If a module calls inter_module_register() on load, it should
> call inter_module_unregister() on unload.  All the registered data
> points into the loaded module, remove the module and the storage
> disappears as well.

You can kmalloc() both the im_name and userdata arguments to
inter_module_register and you ought to be able to pass NULL as the owner.

-- 
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Persistent module storage [was Linux 2.4 Status / TODO page]

[Keith Owens]

On Mon, 6 Nov 2000 00:54:51 +0000 (GMT), 
David Woodhouse <dwmw2@infradead.org> wrote:
>On Mon, 6 Nov 2000, Keith Owens wrote:
>
>> I'm not sure why you think this can be used for module persistent
>> storage.  If a module calls inter_module_register() on load, it should
>> call inter_module_unregister() on unload.  All the registered data
>> points into the loaded module, remove the module and the storage
>> disappears as well.
>
>You can kmalloc() both the im_name and userdata arguments to
>inter_module_register and you ought to be able to pass NULL as the owner.

Ughh!  That is definitely abusing the inter_module functions.  If we
need persistent module storage then we should add a clean interface to
do it instead of using kmalloc and overloading inter_module_xxx.

What do people think, do we need module persistent storage?  There are
already entries in struct module for persistent data but there is no
way of setting those fields, nor is there any code in modutils to
handle persistent storage.  This will probably be a 2.5 change but I
want to get an idea of the requirements before coding anything.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

visual gcc

[Anonymous]

Does anyone know where to find a gui for gcc or g++ or any compiler for a
KDE shell?


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: visual gcc

[Alexander Viro]

On Sun, 5 Nov 2000, Anonymous wrote:

> Does anyone know where to find a gui for gcc or g++ or any compiler for a
> KDE shell?

Yes.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

On Mon, 6 Nov 2000, Keith Owens wrote:

> On Mon, 6 Nov 2000 00:54:51 +0000 (GMT), 
> David Woodhouse <dwmw2@infradead.org> wrote:
> >On Mon, 6 Nov 2000, Keith Owens wrote:
> >
> >> I'm not sure why you think this can be used for module persistent
> >> storage.  If a module calls inter_module_register() on load, it should
> >> call inter_module_unregister() on unload.  All the registered data
> >> points into the loaded module, remove the module and the storage
> >> disappears as well.
> >
> >You can kmalloc() both the im_name and userdata arguments to
> >inter_module_register and you ought to be able to pass NULL as the owner.
> 
> Ughh!  That is definitely abusing the inter_module functions.  If we
> need persistent module storage then we should add a clean interface to
> do it instead of using kmalloc and overloading inter_module_xxx.

Why? It's got to get kmalloc'd anyway, and code reuse is
_good_. Experiment with different names for inter_module_xxx until you
feel happier :)

> What do people think, do we need module persistent storage? 

The primary reason that I've often lamented its removal is for
auto-loaded sound drivers to store their mixer level on unload, in order
to reset to the same values upon being reloaded.

> This will probably be a 2.5 change but I want to get an idea of the
> requirements before coding anything.

Strictly speaking, all the inter_module_xxx stuff should probably wait for
2.5.

-- 
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Oliver Xymoron]

On Mon, 6 Nov 2000, Keith Owens wrote:

> What do people think, do we need module persistent storage?

Hopefully not. The standard examples (mixer levels, etc) are better
handled with a userspace tool hooked by modprobe. This even gets
persistence across reboots if that's what's wanted.

--
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.." 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

On Mon, 6 Nov 2000, Oliver Xymoron wrote:

> Hopefully not. The standard examples (mixer levels, etc) are better
> handled with a userspace tool hooked by modprobe. This even gets
> persistence across reboots if that's what's wanted.

Implement a way for a userspace tool to get the correct mixer levels in
place at the time the sound hardware is reset, so there are no glitches in
the levels, and I'll agree with you.

-- 
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Jeff Garzik]

David Woodhouse wrote:
> 
> On Mon, 6 Nov 2000, Oliver Xymoron wrote:
> 
> > Hopefully not. The standard examples (mixer levels, etc) are better
> > handled with a userspace tool hooked by modprobe. This even gets
> > persistence across reboots if that's what's wanted.
> 
> Implement a way for a userspace tool to get the correct mixer levels in
> place at the time the sound hardware is reset, so there are no glitches in
> the levels, and I'll agree with you.

Linux-Mandrake's initscripts run aumix on bootup and shutdown, to take
care of this...

-- 
Jeff Garzik             | Dinner is ready when
Building 1024           | the smoke alarm goes off.
MandrakeSoft            |	-/usr/games/fortune
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Oliver Xymoron]

On Mon, 6 Nov 2000, David Woodhouse wrote:

> On Mon, 6 Nov 2000, Oliver Xymoron wrote:
> 
> > Hopefully not. The standard examples (mixer levels, etc) are better
> > handled with a userspace tool hooked by modprobe. This even gets
> > persistence across reboots if that's what's wanted.
> 
> Implement a way for a userspace tool to get the correct mixer levels in
> place at the time the sound hardware is reset, so there are no glitches in
> the levels, and I'll agree with you.

If I understand you correctly:

process 1         process 2
open(/dev/dsp)
modprobe->        
                  load module
                  init module   (can't remember which context, actually)
start writing     
                  set mixer levels

Is there any reason we ever want to unblock process 1 before process 2
terminates?

--
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.." 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

On Mon, 6 Nov 2000, Jeff Garzik wrote:

> Linux-Mandrake's initscripts run aumix on bootup and shutdown, to take
> care of this...

So does Red Hat. You can also have a post-install script which does it
after a module is auto-loaded. There can still be a number of seconds
between the initialisation of the hardware and the desired mixer levels
actually getting set.


-- 
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

On Mon, 6 Nov 2000, Oliver Xymoron wrote:

> If I understand you correctly:
> 
> process 1         process 2
> open(/dev/dsp)
> modprobe->        
>                   load module
>                   init module   (can't remember which context, actually)
> start writing     
>                   set mixer levels
> 

> Is there any reason we ever want to unblock process 1 before process 2
> terminates?

No, and I don't think we do. That's not the point.

'init module' is still _after_ 'set mixer levels'. There is a period
during which the mixer levels are changed.

The desired mixer levels should be available to the module at the time of
initialisation.


-- 
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Jeff Garzik]

David Woodhouse wrote:
> The desired mixer levels should be available to the module at the time of
> initialisation.

For drivers built into the kernel that gets messy.  The command line is
only so long.  Sounds messy for modules too.  Further (responding to
your other e-mail), few probably care about having the mixer containing
default, not custom, values for 10 seconds between driver init and aumix
execution from initscripts...

It sounds smarter to delay mixer initialization, or mute all mixer
channels at init.  That effectively initializes the mixer channels to
the custom values you desire, without having to add special case module
gunk for the subset of people who need correct mixer values Right
Now(tm).

	Jeff


-- 
Jeff Garzik             | Dinner is ready when
Building 1024           | the smoke alarm goes off.
MandrakeSoft            |	-/usr/games/fortune
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Oliver Xymoron]

On Mon, 6 Nov 2000, David Woodhouse wrote:

> On Mon, 6 Nov 2000, Oliver Xymoron wrote:
> 
> > If I understand you correctly:
> > 
> > process 1         process 2
...
> 
> > Is there any reason we ever want to unblock process 1 before process 2
> > terminates?
> 
> No, and I don't think we do. That's not the point.
> 
> 'init module' is still _after_ 'set mixer levels'. There is a period
> during which the mixer levels are changed.

Perhaps you mean before? Otherwise you've lost me.

> The desired mixer levels should be available to the module at the time of
> initialisation.

Is this because active audio sources other than /dev/dsp writers are
suddenly in and out of the mix? If there's nothing on the inputs, it
shouldn't matter whether you're changing the levels.

The right way to do this (according to any sound engineer) is to
initialize all the levels to zero unless told otherwise. This would
doubtless annoy the average user, but is more or less equivalent to not
forwarding packets by default.

--
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.." 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

On Mon, 6 Nov 2000, Jeff Garzik wrote:

> David Woodhouse wrote:
> > The desired mixer levels should be available to the module at the time of
> > initialisation.
> 
> For drivers built into the kernel that gets messy.  The command line is
> only so long.  Sounds messy for modules too.  Further (responding to
> your other e-mail), few probably care about having the mixer containing
> default, not custom, values for 10 seconds between driver init and aumix
> execution from initscripts...

I don't mean this to happen on boot. As you say, that gets messy. The
first time a module is loaded after booting, the levels can be all zeroed. 

I'm more interested in the case where the module is loaded for the second
time:

User loads a mixer to set the 'line' level to something sensible so he can
listen to the radio, which is routed through the sound card to his amp.

User closes mixer program. Module is unloaded. Levels remain the same -
all is well.

Some time later, something tries to 'beep' via /dev/audio. Module is
reloaded, the feed the user was listening to is interrupted.

Actually, the way it happened to me was that it was five in the morning, I
was in halls of residence, and suddenly I was playing the radio at full
volume :)

After fixing the sb16 driver to use the existing persistent storage, and
watching that facility disappear from the module support shortly
thereafter, I just decided not to autoload the sound modules.

-- 
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

On Mon, 6 Nov 2000, Oliver Xymoron wrote:

> > 'init module' is still _after_ 'set mixer levels'. There is a period
> > during which the mixer levels are changed.
> 
> Perhaps you mean before? Otherwise you've lost me.

Yeah, sorry, not enough coffee yet this morning.

> > The desired mixer levels should be available to the module at the time of
> > initialisation.
> 
> Is this because active audio sources other than /dev/dsp writers are
> suddenly in and out of the mix? If there's nothing on the inputs, it
> shouldn't matter whether you're changing the levels.

Yes. 

> The right way to do this (according to any sound engineer) is to
> initialize all the levels to zero unless told otherwise. This would
> doubtless annoy the average user, but is more or less equivalent to not
> forwarding packets by default.

The current situation is equivalent to stopping forwarding packets each
time an app on the local machine decides it wants to send its own packets,
after a period of inactivity.

Defaulting to zero on boot is fine. Defaulting to zero after the module
has been auto-unloaded and auto-loaded again is less good.

-- 
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> > Implement a way for a userspace tool to get the correct mixer levels in
> > place at the time the sound hardware is reset, so there are no glitches in
> > the levels, and I'll agree with you.
> 
> Linux-Mandrake's initscripts run aumix on bootup and shutdown, to take
> care of this...

And they don't solve the problem David was talking about. There is a short
deeply unpleasant scream from some soundcards on reload because the card init
and the 0.5-1 second later aumix run dont stop the feedback loop fast enough
when a mic is plugged in

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Dan Hollis]

On Mon, 6 Nov 2000, Alan Cox wrote:
> And they don't solve the problem David was talking about. There is a short
> deeply unpleasant scream from some soundcards on reload because the card init
> and the 0.5-1 second later aumix run dont stop the feedback loop fast enough
> when a mic is plugged in

This is why alsa starts up all devices totally muted. Maybe its time for
David to move to alsa ;)

-Dan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Jeff Garzik]

Dan Hollis wrote:
> 
> On Mon, 6 Nov 2000, Alan Cox wrote:
> > And they don't solve the problem David was talking about. There is a short
> > deeply unpleasant scream from some soundcards on reload because the card init
> > and the 0.5-1 second later aumix run dont stop the feedback loop fast enough
> > when a mic is plugged in
> 
> This is why alsa starts up all devices totally muted. Maybe its time for
> David to move to alsa ;)

I wouldn't mind leaving devices totally muted until open()...

-- 
Jeff Garzik             | Dinner is ready when
Building 1024           | the smoke alarm goes off.
MandrakeSoft            |	-/usr/games/fortune
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

goemon@anime.net said:
>  This is why alsa starts up all devices totally muted. Maybe its time
> for David to move to alsa ;)

Muted is not what I want either, although that's fine when the module is 
_first_ loaded after booting.

What I want is for the mixer settings not to change at all, when the module
is auto-unloaded and later auto-loaded again. I may have set them to pass 
through the line input.

--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Jeff Garzik]

David Woodhouse wrote:
> 
> goemon@anime.net said:
> >  This is why alsa starts up all devices totally muted. Maybe its time
> > for David to move to alsa ;)
> 
> Muted is not what I want either, although that's fine when the module is
> _first_ loaded after booting.
> 
> What I want is for the mixer settings not to change at all, when the module
> is auto-unloaded and later auto-loaded again. I may have set them to pass
> through the line input.

The API allows for setup activity to occur on the fd before sound is
actually started, mixer setup can be one of those steps...

-- 
Jeff Garzik             | Dinner is ready when
Building 1024           | the smoke alarm goes off.
MandrakeSoft            |	-/usr/games/fortune
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Jeff Garzik]

Remember the /dev/mixer and /dev/dsp are separate.

* Driver initializes mixer to 100% muted
* Userspace app sets desired values to /dev/mixer
* Userspace app opens /dev/dsp to play sound

I don't see where any sound can "escape" in this scenario, and it
doesn't require any module data persistence...

	Jeff


-- 
Jeff Garzik             | Dinner is ready when
Building 1024           | the smoke alarm goes off.
MandrakeSoft            |	-/usr/games/fortune
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> > This is why alsa starts up all devices totally muted. Maybe its time for
> > David to move to alsa ;)
> 
> I wouldn't mind leaving devices totally muted until open()...

You need to leave the mixer for cd, tv and radio pass through
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Jeff Garzik]

Alan Cox wrote:
> 
> > > This is why alsa starts up all devices totally muted. Maybe its time for
> > > David to move to alsa ;)
> >
> > I wouldn't mind leaving devices totally muted until open()...
> 
> You need to leave the mixer for cd, tv and radio pass through

Good point.  Also might need to flush default settings to the mixer, if
you start playing w/out first setting the mixer to anything...

-- 
Jeff Garzik             | Dinner is ready when
Building 1024           | the smoke alarm goes off.
MandrakeSoft            |	-/usr/games/fortune
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

jgarzik@mandrakesoft.com said:
>  * Driver initializes mixer to 100% muted * Userspace app sets desired
> values to /dev/mixer * Userspace app opens /dev/dsp to play sound

> I don't see where any sound can "escape" in this scenario, and it
> doesn't require any module data persistence... 


* User boots kernel
* User loads mixer app
* Sound module is autoloaded, defaults to zero levels. This is fine.
* User sets 'line in' level to appropriate level to listen to radio
* User closes mixer app
* Time passes
* Sound module is unloaded
* User continues to happily listen to radio through sound card
* Time passes
* User is listening intently to something on the radio
* Something wants to beep through /dev/audio
* Sound module is autoloaded again, default to zero levels.
	This time it is _NOT_ fine. User is rightly pissed off :)



It's fine to default to zero levels the first time the sound module is 
loaded after booting. Not on subsequent reloads though. 

A long time ago, I made the sb16 driver use the persistent storage facility 
of kerneld to store mixer levels. I was happy with this right up until 
kerneld disappeared :)

I then made a version which read the current mixer levels back from the 
hardware on initialisation, and didn't reset them. That didn't work on all 
sb16 hardware, but it worked for me (tm). 

Persistent storage is the best way to do it though. It doesn't have to be 
persistent over reboots, just during the lifetime of the kernel.

The inter_module_xxx() stuff would facilitate this quite nicely.

--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Jeff Garzik]

David Woodhouse wrote:
> * Sound module is unloaded
> * User continues to happily listen to radio through sound card

You're using the sound card without a driver?


> * Time passes
> * User is listening intently to something on the radio
> * Something wants to beep through /dev/audio
> * Sound module is autoloaded again, default to zero levels.
>         This time it is _NOT_ fine. User is rightly pissed off :)

If you create a post-action in /etc/modules.conf which initializes the
mixer to proper levels, this problem does not exist.

No kernel coding required.

	Jeff


-- 
Jeff Garzik             | Dinner is ready when
Building 1024           | the smoke alarm goes off.
MandrakeSoft            |	-/usr/games/fortune
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

jgarzik@mandrakesoft.com said:
> > * User continues to happily listen to radio through sound card
> You're using the sound card without a driver?

Yes. The sound card allows itself to be unloaded when the pass-through mixer
levels are non-zero. This is reasonable iff it can be reloaded without 
destroying those levels again.

jgarzik@mandrakesoft.com said:
>  If you create a post-action in /etc/modules.conf which initializes
> the mixer to proper levels, this problem does not exist.

Yes it does. It can be a few seconds between initialisation and the 
post-action running. That's plenty of time to miss what the news announcer 
was saying about whether you need to go to work today (my gf is a teacher) 
or to wake the entire house if the mixer levels don't default to zero.

--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Jeff Garzik]

David Woodhouse wrote:
> 
> jgarzik@mandrakesoft.com said:
> > > * User continues to happily listen to radio through sound card
> > You're using the sound card without a driver?
> 
> Yes. The sound card allows itself to be unloaded when the pass-through mixer
> levels are non-zero. This is reasonable iff it can be reloaded without
> destroying those levels again.

I don't think that is reasonable.

The first thing most drivers do is reset the hardware.   That inevitably
leads to some sort of blip, when it comes to sound drivers.  If you
-don't- reset the hardware, the driver is using hardware that is in an
unknown state.  (using hardware w/out resetting it == unknown state)

You are depending on the hardware to keep its state -between- driver
unload and driver reload.  That seems inherently unstable to me.  It
amazes me that such is supported.

	Jeff


-- 
Jeff Garzik             | Dinner is ready when
Building 1024           | the smoke alarm goes off.
MandrakeSoft            |	-/usr/games/fortune
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> I don't think that is reasonable.

I think its totally reasonable.

> The first thing most drivers do is reset the hardware.   That inevitably

Because there is no persistent storage to remember the fact the hardware is
running.

> You are depending on the hardware to keep its state -between- driver
> unload and driver reload.  That seems inherently unstable to me.  It
> amazes me that such is supported.

Well if you want to rewrite the entire module handling and locking so that
mixer levels determine the load/unload behaviour according to AC97 register
combinations and then train users to mute all their inputs before using
rmmod go ahead 8)

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: visual gcc

[Davide Libenzi]

On Mon, 06 Nov 2000, Alexander Viro wrote:
> On Sun, 5 Nov 2000, Anonymous wrote:
> 
> > Does anyone know where to find a gui for gcc or g++ or any compiler for a
> > KDE shell?
> 
> Yes.

:^) 

www.kdevelop.org

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: visual gcc

[Mohammad A. Haque]

Bah!! You ruined the fun! =)

Davide Libenzi wrote:
> 
> On Mon, 06 Nov 2000, Alexander Viro wrote:
> > On Sun, 5 Nov 2000, Anonymous wrote:
> >
> > > Does anyone know where to find a gui for gcc or g++ or any compiler for a
> > > KDE shell?
> >
> > Yes.
> 
> :^)
> 
> www.kdevelop.org

-- 

=====================================================================
Mohammad A. Haque                              http://www.haque.net/ 
                                               mhaque@haque.net

  "Alcohol and calculus don't mix.             Project Lead
   Don't drink and derive." --Unknown          http://wm.themes.org/
                                               batmanppc@themes.org
=====================================================================
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

jgarzik@mandrakesoft.com said:
> > The sound card allows itself to be unloaded when the pass-through
> > mixer levels are non-zero. This is reasonable iff ...

> I don't think that is reasonable. 

You don't think that it's reasonable for the sound card to allow itself to 
be unloaded when the pass-through mixer levels are non-zero? 

So you're suggesting that we should prevent the sound drivers from being 
unloaded at all in that situation?

That would also solve the problem, at the cost of still keeping the sound 
module in unpagable RAM all the time.

jgarzik@mandrakesoft.com said:
> The first thing most drivers do is reset the hardware.   That
> inevitably leads to some sort of blip, when it comes to sound drivers.

A _momentary_ blip on certain hardware is acceptable if it's unavoidable.
Changing the levels and leaving them wrong for seconds before a user-space 
post-install script fixes them is not acceptable.

jgarzik@mandrakesoft.com said:
> You are depending on the hardware to keep its state -between- driver
> unload and driver reload.  That seems inherently unstable to me. 

No we're not. After the kerneld code was removed, I hacked up something to
do that, but it was a suboptimal solution and wasn't reliable on all
hardware. As I said, persistent storage is the better solution.

With persistent storage, the sound driver is free to reset and initialise
the sound card hardware upon reload - it's just that it can initialise it to
the levels which the user had previously set, rather than to the compiled-in
default levels (which are preferably zero). 

Initialising the levels to a default and expecting a user-space app to fix it
later is not good enough.

--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

I think we're getting confused. What I'm advocating is something like this:

init_module()
{
	struct mixer_levels *levels;

	levels = inter_module_get("mysoundcard_mixerlevels");

	if (!levels)
		/* We haven't been loaded before. Default to zero */
		levels = &default_levels;

	init_hardware(levels);
}

cleanup_module()
{
	struct mixer_levels *levels = kmalloc(sizeof *levels);

	if (levels) {
		/* Record current the levels so we can init the hardware
		   to the same next time we're loaded */
		memcpy(levels, current_levels, sizeof(*levels));
		inter_module_register("mysoundcard_mixerlevels", levels);
	}
}


(Note it's pseudocode. I _know_ it doesn't compile and that the name we 
pass to inter_module_register is removed when the module is unloaded. Oh 
and that inter_module_register will panic() and kill the whole system on 
the second unload because a registration with that name already exists.)


--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Mon, 06 Nov 2000, David Woodhouse wrote:
> jgarzik@mandrakesoft.com said:
> > > * User continues to happily listen to radio through sound card
> > You're using the sound card without a driver?
> 
> Yes. The sound card allows itself to be unloaded when the pass-through mixer
> levels are non-zero. This is reasonable iff it can be reloaded without 
> destroying those levels again.
> 
> jgarzik@mandrakesoft.com said:
> >  If you create a post-action in /etc/modules.conf which initializes
> > the mixer to proper levels, this problem does not exist.
> 
> Yes it does. It can be a few seconds between initialisation and the 
> post-action running. That's plenty of time to miss what the news announcer 
> was saying about whether you need to go to work today (my gf is a teacher) 
> or to wake the entire house if the mixer levels don't default to zero.

So autoload the module with a "dont_screw_with_mixer" option. When the kernel
first boots, initialise the mixer to suitable settings (load the module with 
"do_screw_with_mixer" or whatever); thereafter, the driver shouldn't change
the mixer settings on load.


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Jeff Garzik]

David Woodhouse wrote:
> 
> jgarzik@mandrakesoft.com said:
> > > The sound card allows itself to be unloaded when the pass-through
> > > mixer levels are non-zero. This is reasonable iff ...
> 
> > I don't think that is reasonable.
> 
> You don't think that it's reasonable for the sound card to allow itself to
> be unloaded when the pass-through mixer levels are non-zero?
> 
> So you're suggesting that we should prevent the sound drivers from being
> unloaded at all in that situation?

I am thinking about the bigger picture:  You are unloading a driver,
then continuing to use the hardware.  To me, that is an undefined state.


> That would also solve the problem, at the cost of still keeping the sound
> module in unpagable RAM all the time.

Oh, the horror!

[jgarzik@rum linux_2_4]$ ls -l drivers/sound/via82cxxx_audio.o
-rw-r--r--    1 jgarzik  jgarzik     27968 Nov  6 03:28
drivers/sound/via82cxxx_audio.o

So you would rather load everybody's kernel down with mixer level /
module persistence gunk... than simply load a kernel module at boot, and
leave it alone?


> With persistent storage, the sound driver is free to reset and initialise
> the sound card hardware upon reload - it's just that it can initialise it to
> the levels which the user had previously set, rather than to the compiled-in
> default levels (which are preferably zero).
> 
> Initialising the levels to a default and expecting a user-space app to fix it
> later is not good enough.

The one thing that you and I agree on:  It would be nice if the driver
did not init the mixer to a set of defaults, when a preferred set is
available.

However, since simply leaving the driver loaded solves all this mess, it
doesn't seem worth changing drivers to do anything different.

	Jeff


-- 
Jeff Garzik             | "When I do this, my computer freezes."
Building 1024           |          -user
MandrakeSoft            | "Don't do that."
                        |          -level 1
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

jas88@cam.ac.uk said:
>  So autoload the module with a "dont_screw_with_mixer" option. When
> the kernel first boots, initialise the mixer to suitable settings
> (load the module with  "do_screw_with_mixer" or whatever); thereafter,
> the driver shouldn't change the mixer settings on load. 

Not reliable. You can't read back the current mixer state from all 
hardware, even if you _are_ willing to assume that it has remained in a 
sensible state while the driver has been unloaded. 

The driver needs to reset the card to the desired levels. 

--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: visual gcc

[Horst von Brand]

Alexander Viro <viro@math.psu.edu> said:
> On Sun, 5 Nov 2000, Anonymous wrote:
> > Does anyone know where to find a gui for gcc or g++ or any compiler for a
> > KDE shell?

> Yes.

For a _really_ helpful answer, you are missing: "I don't" ;-)
-- 
Dr. Horst H. von Brand                       mailto:vonbrand@inf.utfsm.cl
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Andrew Pimlott]

On Mon, Nov 06, 2000 at 08:00:05AM +0000, David Woodhouse wrote:
> I'm more interested in the case where the module is loaded for the second
> time:

Is there really a reason to unload a module in normal usage?  Beyond
miniscule memory savings and hack value?  You can solve the whole
problem with a loud "don't do that".

Andrew
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

jgarzik@mandrakesoft.com said:
>  I am thinking about the bigger picture:  You are unloading a driver,
> then continuing to use the hardware.  To me, that is an undefined
> state.

We're only using the pass-through levels. It's undefined but it doesn't
matter to the software. I'd actually suggest that for hardware which does
stop the pass-through audio when the driver is unloaded, we really ought not
unload the driver while those levels are non-zero.

We should still reset the hardware completely when we reload the driver -
it's just that we should reset it to the levels previously set by the user,
rather than resetting it to zeroes.


jgarzik@mandrakesoft.com said:
>  However, since simply leaving the driver loaded solves all this mess,
> it doesn't seem worth changing drivers to do anything different. 

Leaving the driver loaded has been my solution ever since kerneld was taken 
out. I merely commented that Keith's new stuff would allow me to get 
persistent storage working again. It's not very difficult to change the 
driver to use it.

I believe the SBLive! driver is a little larger than the example you 
pasted. I think we probably do care about adding a little bit more to the 
pool of permanently unswappable pages here and there.


--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Martin Dalecki]

David Woodhouse wrote:
> 
> jgarzik@mandrakesoft.com said:
> >  * Driver initializes mixer to 100% muted * Userspace app sets desired
> > values to /dev/mixer * Userspace app opens /dev/dsp to play sound
> 
> > I don't see where any sound can "escape" in this scenario, and it
> > doesn't require any module data persistence...
> 
> * User boots kernel
> * User loads mixer app
> * Sound module is autoloaded, defaults to zero levels. This is fine.
> * User sets 'line in' level to appropriate level to listen to radio
> * User closes mixer app
> * Time passes
> * Sound module is unloaded
> * User continues to happily listen to radio through sound card
> * Time passes
> * User is listening intently to something on the radio
> * Something wants to beep through /dev/audio
> * Sound module is autoloaded again, default to zero levels.
>         This time it is _NOT_ fine. User is rightly pissed off :)
> 
> It's fine to default to zero levels the first time the sound module is
> loaded after booting. Not on subsequent reloads though.
> 
> A long time ago, I made the sb16 driver use the persistent storage facility
> of kerneld to store mixer levels. I was happy with this right up until
> kerneld disappeared :)

It was removed for good reasons.

> I then made a version which read the current mixer levels back from the
> hardware on initialisation, and didn't reset them. That didn't work on all
> sb16 hardware, but it worked for me (tm).
> 
> Persistent storage is the best way to do it though. It doesn't have to be
> persistent over reboots, just during the lifetime of the kernel.

No! The best way to do it are just *persistently loaded* modules.
It's THAT simple!
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Mon, 06 Nov 2000, David Woodhouse wrote:
> jas88@cam.ac.uk said:
> >  So autoload the module with a "dont_screw_with_mixer" option. When
> > the kernel first boots, initialise the mixer to suitable settings
> > (load the module with  "do_screw_with_mixer" or whatever); thereafter,
> > the driver shouldn't change the mixer settings on load. 
> 
> Not reliable. You can't read back the current mixer state from all 
> hardware, even if you _are_ willing to assume that it has remained in a 
> sensible state while the driver has been unloaded. 

Irrelevant. The current mixer settings don't matter: what matters is that the
driver does not change them.

> The driver needs to reset the card to the desired levels. 

What desired levels? The only desired levels are the current ones, which
the driver does not and (sometimes) cannot know. Leave well alone.


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

jas88@cam.ac.uk said:
>  Irrelevant. The current mixer settings don't matter: what matters is
> that the driver does not change them.

It does matter. The sound driver needs to be able to _read_ the current 
levels. Almost all mixer programs will start by doing this, to set the 
slider to the correct place.

> > The driver needs to reset the card to the desired levels. 

> What desired levels? The only desired levels are the current ones,
> which the driver does not and (sometimes) cannot know. Leave well
> alone.

It does not know them. Correct. But with persistent module storage, it 
_could_ know them. It cannot know them the _first_ time the module is 
loaded after booting. That's fine. On subsequent loads, it can and 
should DTRT.

--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Horst von Brand]

David Woodhouse <dwmw2@infradead.org> said:
> jas88@cam.ac.uk said:
> >  Irrelevant. The current mixer settings don't matter: what matters is
> > that the driver does not change them.

> It does matter. The sound driver needs to be able to _read_ the current 
> levels. Almost all mixer programs will start by doing this, to set the 
> slider to the correct place.

OK, how then using _2_ modules, data and worker:

- Data (containing the mixer levels or whatever other data you want to save)
  can only be unloaded after resetting to some default state. When loading
  it sets the default state.
- Worker does the work, and on loading loads the data one (if not yet
  resident) [This is automatic as the worker depends on symbols the data
  module exports].

No funny "persistent data" mechanisms or screwups when the worker gets
removed and reinserted. In many cases the data module could be shared among
several others, in other cases it would have to be able lo load several
times or manage several incarnations of its payload.

Sure, makes sense only if the worker module is largeish; if not, just let
it stay put (When reconfiguring anything, increment module use count;
decrement on reset. This should do the trick also for the data module
above).
-- 
Dr. Horst H. von Brand                       mailto:vonbrand@inf.utfsm.cl
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Mon, 06 Nov 2000, David Woodhouse wrote:
> jas88@cam.ac.uk said:
> >  Irrelevant. The current mixer settings don't matter: what matters is
> > that the driver does not change them.
> 
> It does matter. The sound driver needs to be able to _read_ the current 
> levels.

So do so. That's a hardware/driver issue. If the hardware is broken, put
a workaround in the driver for that hardware (make the driver persistent,
as Horst suggested, perhaps). Don't kludge the kernel to mask hardware
bugs.

> Almost all mixer programs will start by doing this, to set the 
> slider to the correct place.

Yippee. As we all know, implementing GUI volume controls
and putting the slider in the right place is a kernel function,
and nothing to do with userspace...

If you want your volume control applet to be able to read the
current volume settings, even on buggy hardware which can't
really do that, put the kludge in userspace. Or if you really want,
put it in your driver for buggy hardware, in the way Horst suggested.

> > > The driver needs to reset the card to the desired levels. 
> 
> > What desired levels? The only desired levels are the current ones,
> > which the driver does not and (sometimes) cannot know. Leave well
> > alone.
> 
> It does not know them. Correct. But with persistent module storage, it 
> _could_ know them.

No it cannot. The desired levels have not been defined: there are no
desired levels to determine! Don't tamper with settings you don't need
to. 

> It cannot know them the _first_ time the module is 
> loaded after booting. That's fine. On subsequent loads, it can and 
> should DTRT.

The right thing in this context is not to screw with hardware settings
unless and until it is given settings to set. Do not set values arbitrarily:
set only the values you are explicitly given. Anything else is simply
a bug in your driver.


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Horst von Brand]

[Chopped down Cc: list]
"James A. Sutherland" <jas88@cam.ac.uk> said:
> On Mon, 06 Nov 2000, David Woodhouse wrote:

[...]

> > It does not know them. Correct. But with persistent module storage, it 
> > _could_ know them.

> No it cannot. The desired levels have not been defined: there are no
> desired levels to determine! Don't tamper with settings you don't need
> to. 

The problem (AFAIU) is that if the levels aren't set on startup, they are
random in some cases. So you'd have to save (at least) the fact that they
have been initalized. Just that would be easy: Set aside a word in the
kernel, which is set to 0 when booting, and which then gets the value 1
when the hardware is initialized. For more fancy stuff, splitting the
module into data/code (as I suggested) should do the trick with minimal
impact on the rest.
-- 
Dr. Horst H. von Brand                       mailto:vonbrand@inf.utfsm.cl
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Mon, 06 Nov 2000, Horst von Brand wrote:
> [Chopped down Cc: list]
> "James A. Sutherland" <jas88@cam.ac.uk> said:
> > On Mon, 06 Nov 2000, David Woodhouse wrote:
> 
> [...]
> 
> > > It does not know them. Correct. But with persistent module storage, it 
> > > _could_ know them.
> 
> > No it cannot. The desired levels have not been defined: there are no
> > desired levels to determine! Don't tamper with settings you don't need
> > to. 
> 
> The problem (AFAIU) is that if the levels aren't set on startup, they are
> random in some cases.

So set them on startup. NOT when the driver is first loaded. Put it in
the rc.d scripts.

> So you'd have to save (at least) the fact that they
> have been initalized. 

No, you don't.

> Just that would be easy: Set aside a word in the
> kernel, which is set to 0 when booting, and which then gets the value 1
> when the hardware is initialized.

Why bother? Just set the settings *explicitly* on boot, rather than in the
driver itself.

> For more fancy stuff, splitting the
> module into data/code (as I suggested) should do the trick with minimal
> impact on the rest.

No need. Let userspace save it somewhere, if that's needed.


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

vonbrand@inf.utfsm.cl said:
>  No funny "persistent data" mechanisms or screwups when the worker
> gets removed and reinserted. In many cases the data module could be
> shared among several others, in other cases it would have to be able
> lo load several times or manage several incarnations of its payload. 

The reason I brought this up now is because Keith's new 
inter_module_register stuff could easily be used to provide this 
functionality _without_ the extra module remaining loaded.


--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

jas88@cam.ac.uk said:
>  Yippee. As we all know, implementing GUI volume controls and putting
> the slider in the right place is a kernel function, and nothing to do
> with userspace... 

Don't troll, James. The kernel needs to provide the functionality required 
by userspace. The functionality required in this case is the facility to 
read the current mixer levels.


jas88@cam.ac.uk said:
>  The right thing in this context is not to screw with hardware
> settings unless and until it is given settings to set. Do not set
> values arbitrarily: set only the values you are explicitly given.
> Anything else is simply a bug in your driver. 

It is unwise to assume that the hardware is in a sane state when the driver 
has been unloaded and reloaded. I agree that you should set the values that 
were explicitly given. That's why we should remember them.

--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

jas88@cam.ac.uk said:
>  So set them on startup. NOT when the driver is first loaded. Put it
> in the rc.d scripts. 

No. You should initialise the hardware completely when the driver is 
reloaded. Although the expected case is that the levels just happen to be 
the same as the last time the module was loaded, you can't know that the 
machine hasn't been suspended and resumed since then.


jas88@cam.ac.uk said:
>  No need. Let userspace save it somewhere, if that's needed. 

Don't troll, James. Reread the thread and see why doing it in userspace is 
too late.

--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> So autoload the module with a "dont_screw_with_mixer" option. When the kernel
> first boots, initialise the mixer to suitable settings (load the module with 
> "do_screw_with_mixer" or whatever); thereafter, the driver shouldn't change
> the mixer settings on load.

Which is part of what persistent module data lets you do. And without having
to mess with dont_screw_with_mixer (which if you get it wrong btw can be 
fatal and hang the hardware)


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> > Persistent storage is the best way to do it though. It doesn't have to be
> > persistent over reboots, just during the lifetime of the kernel.
> 
> No! The best way to do it are just *persistently loaded* modules.
> It's THAT simple!

So you want to split every sound driver into two or more modules ? 

Alan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> No funny "persistent data" mechanisms or screwups when the worker gets
> removed and reinserted. In many cases the data module could be shared among
> several others, in other cases it would have to be able lo load several
> times or manage several incarnations of its payload.

It actually seems the persistent data mechanism in user space wouldnt be
much different in implementation.

Add a 'preserved' tag for one section of module memory. On load look up the
data, if its from this boot memcpy it into the module. On unload write it
back to disk. No kernel code needed.

Alan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alon Ziv]

The best solution to the sound driver issue, IMHO, is still entirely
userspace---
just no-one has written it yet.
What we should do:
1. Before auto-unload of the driver, run a small utility which will read
mixer settings
   and save them somewhere
2. When auto-loading the driver, use driver arguments which are initialized
from the
   settings saved above
All that's missing is the method of passing data from step 1 to step 2.

----- Original Message -----
From: "David Woodhouse" <dwmw2@infradead.org>
To: "Horst von Brand" <vonbrand@inf.utfsm.cl>
Cc: <linux-kernel@vger.kernel.org>
Sent: Monday, November 06, 2000 19:06
Subject: Re: Persistent module storage [was Linux 2.4 Status / TODO page]


>
> vonbrand@inf.utfsm.cl said:
> >  No funny "persistent data" mechanisms or screwups when the worker
> > gets removed and reinserted. In many cases the data module could be
> > shared among several others, in other cases it would have to be able
> > lo load several times or manage several incarnations of its payload.
>
> The reason I brought this up now is because Keith's new
> inter_module_register stuff could easily be used to provide this
> functionality _without_ the extra module remaining loaded.
>
>
> --
> dwmw2
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
>

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

alonz@usa.net said:
> The best solution to the sound driver issue, IMHO, is still entirely
> userspace--- just no-one has written it yet. What we should do: 1.
> Before auto-unload of the driver, run a small utility which will read
> mixer settings
>    and save them somewhere 2. When auto-loading the driver, use driver
> arguments which are initialized from the
>    settings saved above

That could work, although it may be better to make it more generic and 
capable of handling any form of data. 

Any form of persistent storage would do - and if it can be handled entirely
in userspace, all the better. I merely pointed out that Keith's 
inter_module_xxx could provide this quite cleanly. Others disputed that it 
was required at all.

--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> Drivers are supposed to handle present hardware - if the hardware
> is there - there should be a driver handling it as well.

Thats not how things have worked historically. Thats not consistent with other
modules either.

> The argument for saving some memmory is nonapplicable becouse in
> the case of expected usage in the future you have anyway to assume that
> in this futere there will be sufficient memmory for it there. And then

Rubbish

> Could some one add this to the FAQ ... please!

You got the letters in the wrong order. Your proposal is at best a 
Frequently Questioned Answer

> 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Mon, 06 Nov 2000, David Woodhouse wrote:
> jas88@cam.ac.uk said:
> >  Yippee. As we all know, implementing GUI volume controls and putting
> > the slider in the right place is a kernel function, and nothing to do
> > with userspace... 
> 
> Don't troll, James. The kernel needs to provide the functionality required 
> by userspace. The functionality required in this case is the facility to 
> read the current mixer levels.

Except this isn't possible with the hardware in question! If it were, there
would be no problem. In cases where the hardware doesn't support the
functionality userspace "needs", why put the kludge in the kernel?

If userspace wants to know what settings it set last time, it should store
those values somewhere.

> jas88@cam.ac.uk said:
> >  The right thing in this context is not to screw with hardware
> > settings unless and until it is given settings to set. Do not set
> > values arbitrarily: set only the values you are explicitly given.
> > Anything else is simply a bug in your driver. 
> 
> It is unwise to assume that the hardware is in a sane state when the driver 
> has been unloaded and reloaded. I agree that you should set the values that 
> were explicitly given. That's why we should remember them.

No values are being explicitly given. Loading the driver should not cause
any settings to be changed: changing the settings should do that!

There is no need for the drivers to change any settings. If the settings need
to be (re)set, let userspace do it.
 

James. 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> 1. Before auto-unload of the driver, run a small utility which will read
> mixer settings
>    and save them somewhere
> 2. When auto-loading the driver, use driver arguments which are initialized
> from the
>    settings saved above
> All that's missing is the method of passing data from step 1 to step 2.

A simple more generic solution is to do this


struct things_to_keep my_bits
{
	..
};

struct things_to_keep __persistent card_info[NUM_CARDS]
{
}

and have insmod do

	load module up
	open /var/run/moduledata/$modname
	if exists && is from this boot then && is right size
		read data into __persistent ELF section
	endif
	load into kernel
	init module

and rmmod
	cleanup module
	open /var/run/moduledata/$modname
	write data from __persistent segment into file
	
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

alan@lxorguk.ukuu.org.uk said:
> > No! The best way to do it are just *persistently loaded* modules.
> > It's THAT simple!
>
 So you want to split every sound driver into two or more modules ? 

The point here is that although I've put up with just keeping the sound 
driver loaded for the last few years, permanently taking up a large amount 
of DMA memory, the inter_module_xxx stuff that Keith is proposing would 
give us a simple way of storing the data which we want to store. 

It's even simpler (and cleaner) than having to split all the sound drivers 
up into data and worker modules.

Someone suggested combining the 'data' modules so that one data module was 
shared by different 'worker' modules.

Build that into the kernel rather than making it a module.

Call its functions 'inter_module_get' et al.

We seem to have returned to what I was suggesting in the first place.

Being able to do it completely in userspace would be neater, though.

--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Mon, 06 Nov 2000, Alan Cox wrote:
> > So autoload the module with a "dont_screw_with_mixer" option. When the kernel
> > first boots, initialise the mixer to suitable settings (load the module with 
> > "do_screw_with_mixer" or whatever); thereafter, the driver shouldn't change
> > the mixer settings on load.
> 
> Which is part of what persistent module data lets you do. And without having
> to mess with dont_screw_with_mixer (which if you get it wrong btw can be 
> fatal and hang the hardware)

Eh? You just load the driver once, probably on boot, to configure sane values.
This time round, you use an argument (or an ioctl or whatever) to specify the
values you want. (cat /etc/sysconfig/sound/defaultvolume > /dev/sound/mixer or
whatever). After that, the module can be unloaded and loaded as needed, without
any need to touch the mixer settings except in response to *explicit* "set
volume" commands from userspace.


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[David Woodhouse]

jas88@cam.ac.uk said:
>  Except this isn't possible with the hardware in question! If it were,
> there would be no problem. In cases where the hardware doesn't support
> the functionality userspace "needs", why put the kludge in the kernel?

> If userspace wants to know what settings it set last time, it should
> store those values somewhere.

No. You have to reset the hardware fully each time you load the module. 
Although you _expect_ it to be in the state in which you left it, you can't 
be sure of that. 


jas88@cam.ac.uk said:
> Eh? You just load the driver once, probably on boot, to configure sane
> values. This time round, you use an argument (or an ioctl or whatever)
> to specify the values you want. (cat /etc/sysconfig/sound/
> defaultvolume > /dev/sound/mixer or whatever). After that, the module
> can be unloaded and loaded as needed, without any need to touch the
> mixer settings except in response to *explicit* "set volume" commands
> from userspace. 

Agreed. Where 'whatever' == persistent storage of some form. I care not 
what form that takes. If you can store the data entirely in userspace and 
still have them present at the time the driver initialises the hardware, 
that's fine. 


--
dwmw2


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Mon, 06 Nov 2000, David Woodhouse wrote:
> jas88@cam.ac.uk said:
> >  So set them on startup. NOT when the driver is first loaded. Put it
> > in the rc.d scripts. 
> 
> No. You should initialise the hardware completely when the driver is 
> reloaded. Although the expected case is that the levels just happen to be 
> the same as the last time the module was loaded, you can't know that the 
> machine hasn't been suspended and resumed since then.

If suspend/resume changes the settings of the card, you need to deal with that
as a separate issue - otherwise resume isn't restoring things properly. Perhaps
you need to prevent module unloading. Just restoring the correct settings when
the driver is loaded is definitely too late.

> jas88@cam.ac.uk said:
> >  No need. Let userspace save it somewhere, if that's needed. 
> 
> Don't troll, James. Reread the thread and see why doing it in userspace is 
> too late.

Why is it too late? There is no need for the driver to set *any* volume levels
on load. When told to load the driver, just LOAD the DRIVER. Don't reset the
card, or make ANY changes.


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Mon, 06 Nov 2000, David Woodhouse wrote:
> jas88@cam.ac.uk said:
> >  Except this isn't possible with the hardware in question! If it were,
> > there would be no problem. In cases where the hardware doesn't support
> > the functionality userspace "needs", why put the kludge in the kernel?
> 
> > If userspace wants to know what settings it set last time, it should
> > store those values somewhere.
> 
> No. You have to reset the hardware fully each time you load the module. 
> Although you _expect_ it to be in the state in which you left it, you can't 
> be sure of that. 

If a reset is needed, I think it should come explicitly from userspace.

> jas88@cam.ac.uk said:
> > Eh? You just load the driver once, probably on boot, to configure sane
> > values. This time round, you use an argument (or an ioctl or whatever)
> > to specify the values you want. (cat /etc/sysconfig/sound/
> > defaultvolume > /dev/sound/mixer or whatever). After that, the module
> > can be unloaded and loaded as needed, without any need to touch the
> > mixer settings except in response to *explicit* "set volume" commands
> > from userspace. 
> 
> Agreed. Where 'whatever' == persistent storage of some form. I care not 
> what form that takes. If you can store the data entirely in userspace and 
> still have them present at the time the driver initialises the hardware, 
> that's fine. 

That's what I've been getting at all along...

Probably have a simple load and unload script, which dumps state to a file
on unload, and restores it on load. Whenever the module's state is not saved,
the refcount is >0, so you can't unload it without saving state.


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Martin Dalecki]

Horst von Brand wrote:
> 
> David Woodhouse <dwmw2@infradead.org> said:
> > jas88@cam.ac.uk said:
> > >  Irrelevant. The current mixer settings don't matter: what matters is
> > > that the driver does not change them.
> 
> > It does matter. The sound driver needs to be able to _read_ the current
> > levels. Almost all mixer programs will start by doing this, to set the
> > slider to the correct place.
> 
> OK, how then using _2_ modules, data and worker:

People that's all designing for the sake of design.
And it's trying to solve a non existant problem:
Just load the damn module once and leave it there where it is.
Drivers are supposed to handle present hardware - if the hardware
is there - there should be a driver handling it as well.
The argument for saving some memmory is nonapplicable becouse in
the case of expected usage in the future you have anyway to assume that
in this futere there will be sufficient memmory for it there. And then
please remember that all possible space savings are in the granularity
of
pages!

Could some one add this to the FAQ ... please!
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Eric W. Biederman]

David Woodhouse <dwmw2@infradead.org> writes:

> The current situation is equivalent to stopping forwarding packets each
> time an app on the local machine decides it wants to send its own packets,
> after a period of inactivity.
> 
> Defaulting to zero on boot is fine. Defaulting to zero after the module
> has been auto-unloaded and auto-loaded again is less good.

Well we don't have auto unload.
And module persistent data for the second load case causes chaos with 
the goal of having exactly the same code in modules and compiled in
kernel code.

It would probably be better (in this case) to increment the module count
when the mixer settings go above 0, and decrement it when the settings 
go totally to 0.  This prevents an unwanted unload.

But for reliability and code simplicity there does not yet seem to
be a case for persistent module storage.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Paul Jakma]

On Mon, 6 Nov 2000, David Woodhouse wrote:


> * Sound module is autoloaded again, default to zero levels.

so you use the 'post-install' option of modules.conf to run your
mixer-level setting script.

> 	This time it is _NOT_ fine. User is rightly pissed off :)
> 

even better: is there any pressing need for /all/ modules to be
auto-unloaded? things like sound modules should be statically loaded
at boot time and never removed for 99% of workstations i think.

i'd also like to see dist's adopt a nice config file specifying which
modules to statically load at boot-time. (i know i want i them
loaded). then maybe info from depmod could be applied to remove
redundant loads.

> The inter_module_xxx() stuff would facilitate this quite nicely.
> 

but if userspace can do it just as easily... (in the case of sound
modules anyway)

> --
> dwmw2

--paulj

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Oliver Xymoron]

On Mon, 6 Nov 2000, David Woodhouse wrote:

> The point here is that although I've put up with just keeping the sound 
> driver loaded for the last few years, permanently taking up a large amount 
> of DMA memory, the inter_module_xxx stuff that Keith is proposing would 
> give us a simple way of storing the data which we want to store. 
...
> Being able to do it completely in userspace would be neater, though.

I think there are a bunch of other devices that need stuff from userspace
before they fully init, namely the ones that load proprietary firmware
images. Will an approach like that work here?

Alternately, we could have a "virtual module" called mixer-settings-n,
which when requested would tell modprobe to call a program to do its
fiddly things but wouldn't actually load a module.

The virtual module idea is ugly, yes, and perhaps what's needed is a more
generic userspace hook interface. Something that merged what
/sbin/hotplug, /sbin/modprobe, and the much-talked-about devfsd-like
notifier are supposed to do.

--
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.." 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Paul Jakma]

On Mon, 6 Nov 2000, David Woodhouse wrote:

> No. You should initialise the hardware completely when the driver is 
> reloaded.

and it is. just that 'mixer levels' are subjective - different users
have different tastes. in what way does:

- init to mute
- user set to liking

fail people? 

(sound modules shouldn't be unloaded as a matter of course, and user
set can be done with post-install option of modules.conf)

> the same as the last time the module was loaded, you can't know that the 
> machine hasn't been suspended and resumed since then.
> 

reloading modules may be a neccessary hack at present to reinit the
drivers after suspend/resume, but surely the correct answer there is
to go fix power management. the modules are not unloaded by the kernel
as part of the suspend event and they are not loaded as part of the
resume event. the module persists across the power event, so if
something breaks, go fix the power management handling - the driver
doesn't handle it properly!

--paulj

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Jeff Garzik]

Oliver Xymoron wrote:
> 
> On Mon, 6 Nov 2000, David Woodhouse wrote:
> 
> > The point here is that although I've put up with just keeping the sound
> > driver loaded for the last few years, permanently taking up a large amount
> > of DMA memory, the inter_module_xxx stuff that Keith is proposing would
> > give us a simple way of storing the data which we want to store.
> ...
> > Being able to do it completely in userspace would be neater, though.
> 
> I think there are a bunch of other devices that need stuff from userspace
> before they fully init, namely the ones that load proprietary firmware
> images. Will an approach like that work here?

Some devices have a firmware.h that is compiled into the driver.  A few
sound devices use a function that loads a firmware file from userspace,
given a filename.  The comment in drivers/sound/sound_firmware.c says
that this is a poor method, and that the recommended method for
uploading firmware to a device is via ioctl.

	Jeff


-- 
Jeff Garzik             | "When I do this, my computer freezes."
Building 1024           |          -user
MandrakeSoft            | "Don't do that."
                        |          -level 1
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Paul Jakma]

On Mon, 6 Nov 2000, Alan Cox wrote:

> Which is part of what persistent module data lets you do. And without having
> to mess with dont_screw_with_mixer (which if you get it wrong btw can be 
> fatal and hang the hardware)
> 

the sound card case for persistent modules is contentious i think.

what other good reasons are there for persistent data?

--paulj

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Dan Hollis]

On Mon, 6 Nov 2000, James A. Sutherland wrote:
> So autoload the module with a "dont_screw_with_mixer" option. When the kernel
> first boots, initialise the mixer to suitable settings (load the module with 
> "do_screw_with_mixer" or whatever); thereafter, the driver shouldn't change
> the mixer settings on load.

You are asking for real trouble on hotplug hardware if you insist on this.

-Dan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Oliver Xymoron]

On Mon, 6 Nov 2000, Jeff Garzik wrote:

> Oliver Xymoron wrote:
> > 
> > On Mon, 6 Nov 2000, David Woodhouse wrote:
> > 
> > > The point here is that although I've put up with just keeping the sound
> > > driver loaded for the last few years, permanently taking up a large amount
> > > of DMA memory, the inter_module_xxx stuff that Keith is proposing would
> > > give us a simple way of storing the data which we want to store.
> > ...
> > > Being able to do it completely in userspace would be neater, though.
> > 
> > I think there are a bunch of other devices that need stuff from userspace
> > before they fully init, namely the ones that load proprietary firmware
> > images. Will an approach like that work here?
> 
> Some devices have a firmware.h that is compiled into the driver.  A few
> sound devices use a function that loads a firmware file from userspace,
> given a filename.  The comment in drivers/sound/sound_firmware.c says
> that this is a poor method, and that the recommended method for
> uploading firmware to a device is via ioctl.

Ioctl (or alternate device for plan9 groupies) is fine. My point is final
initialization of the device is obviously delayed until the firmware is
loaded. Adopting a similar strategy for initializing mixers (possibly
falling back to initializing with zero levels) minimizes the window
between resetting a device and having sane mixer settings.

--
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.." 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Tim Riker]

Could we handle this by setting a nomixerreset=1 option in modules.conf
or as the module default that would effect module reloads. Then on boot
up explicitly modprobe with a mixerlevel=0 and then set the levels via
aumix etc?

This way the existing hardware can keep the levels if it knows how. As
mentioned there would also have to be a setlevels user script called
after suspend/resume.

Barring this, we could create a persistantdata module that we can
modprobe and then discover with Keith's inter_module_xxx and read/write
opaque data to/from. Then if the user does not want to use it they can
just "alias persistantdata off" it and poof.

David Woodhouse wrote:
> 
> alonz@usa.net said:
> > The best solution to the sound driver issue, IMHO, is still entirely
> > userspace--- just no-one has written it yet. What we should do: 1.
> > Before auto-unload of the driver, run a small utility which will read
> > mixer settings
> >    and save them somewhere 2. When auto-loading the driver, use driver
> > arguments which are initialized from the
> >    settings saved above
> 
> That could work, although it may be better to make it more generic and
> capable of handling any form of data.
> 
> Any form of persistent storage would do - and if it can be handled entirely
> in userspace, all the better. I merely pointed out that Keith's
> inter_module_xxx could provide this quite cleanly. Others disputed that it
> was required at all.
> 
> --
> dwmw2
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/

-- 
Tim Riker - http://rikers.org/ - short SIGs! <g>
All I need to know I could have learned in Kindergarten
... if I'd just been paying attention.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Paul Gortmaker]

Jeff Garzik wrote:
> 
> Alan Cox wrote:
> > >      * Check all devices use resources properly (Everyone now has to use
> > >        request_region and check the return since we no longer single
> > >        thread driver inits in all module cases. Also memory regions are
> > >        now requestable and a lot of old drivers dont know this yet. --
> > >        Alan Cox)
> >
> > Folks have done most of the common drivers. So its not really a show stopper
> > now just a 'clean up'
> 
> s/check_region/request_region/ is moving along, but there is still a
> ways to go.  I agree with "folks have done most of the common drivers"
> 
> I have seen very few additions of request_mem_region, though.

Something which I forgot to add when mentioning my ioremap patch is
that you can pretty much forget about adding request_mem_region() to the
same drivers since both changes are in the same spot of code and doing
one without doing the other is kind of silly. (i.e. the patches I have
for 8390 based drivers do both at once).

Although simply having a dev->vmem [or dev->base] added to struct
netdevice in 2.4.0, as was discussed many months ago(*) might go a long
way in allowing driver back-compatibility from 2.5.x into 2.4.x - probably
worthwhile.

(*) Search <Pine.LNX.4.10.9912281026030.1294-100000@penguin.transmeta.com>
    in any linux-net mailing list archive of Dec 1999.

Paul.



_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Rogier Wolff]

Alan Cox wrote:
> A simple more generic solution is to do this

[....]

> 	if exists && is from this boot then && is right size
> 		read data into __persistent ELF section
> 	endif

Alan, why are you stating "if it's from this boot"? I can think that
maybe you want to keep stuff across boots too. Maybe once we're at it,
have two sections. One that is persistent across boots, the other
isn't.

				Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
*       Common sense is the collection of                                *
******  prejudices acquired by age eighteen.   -- Albert Einstein ********
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Evan Jeffrey]

> On Mon, 06 Nov 2000, David Woodhouse wrote:
> > 
> > No. You have to reset the hardware fully each time you load the module. 
> > Although you _expect_ it to be in the state in which you left it, you can't
>  
> > be sure of that. 
> 
> If a reset is needed, I think it should come explicitly from userspace.
 
Take Alan's example of a USB device, say USB audio.  If it is disconnected
and reconnected to add a hub, or anything else, the device may shut itself
down, go to an undefined state, or even power cycle (if it draws power from 
the USB +5V).  Same with hot-swap PCI cards.  The driver *MUST* reset the 
device on load.  If saving mixer levels through this kind of transition is 
desired (which it evidentally is), the module load/unload code must save and 
restore the settings.

This is exactly equivelent to reseting hardware after a warm boot.  Who knows
what the Windows driver did to your card in the mean time.  A device driver
can only guarantee that nobody horkes with its hardware while it is loaded--
In the interim, the driver may have been connected to another computer,
accessed by another driver, or accessed from userspace (say, VMWare doing
a pass through driver).

I personally like the idea of having insmod/rmmod do copy-out/copy-in from
a cache file in userspace.  That way, if we need large data sets (a ROM
image for something.)  they don't take up kernel space when not in use.
Also, it allows people to have persistant settings across reboot through
the same mechanism--avoiding duplicating information in shutdown/startup 
scripts.

Evan
---
Fear is the mind killer.   -- Frank Herbert, "Dune"
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> It would probably be better (in this case) to increment the module count
> when the mixer settings go above 0, and decrement it when the settings 
> go totally to 0.  This prevents an unwanted unload.

Thats about 200 lines of code and also about 50,000 emails complaining people
cannot unload sound stuff.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> i'd also like to see dist's adopt a nice config file specifying which
> modules to statically load at boot-time. (i know i want i them
> loaded). then maybe info from depmod could be applied to remove
> redundant loads.

Its called modules.conf. It has all these nice preload directives in it

> 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> Some devices have a firmware.h that is compiled into the driver.  A few
> sound devices use a function that loads a firmware file from userspace,
> given a filename.  The comment in drivers/sound/sound_firmware.c says
> that this is a poor method, and that the recommended method for
> uploading firmware to a device is via ioctl.

modutils has a post-install facility that supports firmware load via ioctl
beautifully too

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> the sound card case for persistent modules is contentious i think.
> what other good reasons are there for persistent data?

Maintaining TV tuning
Maintaining configuration options for USB
Maintaining radio tuner frequencies
Speeding up module reloading (avoiding expensive re-inits - eg 30 seconds for
i2o_scsi)
Remembering IDE tuning options across ide module unloads
Avoiding rescanning the scsi bus on a scsi module reload
...


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> Barring this, we could create a persistantdata module that we can
> modprobe and then discover with Keith's inter_module_xxx and read/write
> opaque data to/from. Then if the user does not want to use it they can
> just "alias persistantdata off" it and poof.

All of this is kernel code we don't need.  Its not a good solution generically
or otherwise. Modutils should just use the persistent data stuff that has hooks
ready to roll already.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> > 	if exists && is from this boot then && is right size
> > 		read data into __persistent ELF section
> > 	endif
> 
> Alan, why are you stating "if it's from this boot"? I can think that
> maybe you want to keep stuff across boots too. Maybe once we're at it,
> have two sections. One that is persistent across boots, the other
> isn't.

Possibly. The cases brought up so far are all 'within one boot'. But I agree
you could add long term persistent data if there was a need

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Paul Jakma]

On Mon, 6 Nov 2000, Alan Cox wrote:
> Its called modules.conf. It has all these nice preload directives in it

cool..

doesn't seem to be documented though in modutils 2.3.17. what exactly
does it do?

regards,
-- 
Paul Jakma	paul@clubi.ie
PGP5 key: http://www.clubi.ie/jakma/publickey.txt
-------------------------------------------
Fortune:
I finally went to the eye doctor.  I got contacts.  I only need them to
read, so I got flip-ups.
		-- Steven Wright

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Gerhard Mack]

On Mon, 6 Nov 2000, James A. Sutherland wrote:

> Except this isn't possible with the hardware in question! If it were, there
> would be no problem. In cases where the hardware doesn't support the
> functionality userspace "needs", why put the kludge in the kernel?
> 
> If userspace wants to know what settings it set last time, it should store
> those values somewhere.
> 
> > jas88@cam.ac.uk said:
> > >  The right thing in this context is not to screw with hardware
> > > settings unless and until it is given settings to set. Do not set
> > > values arbitrarily: set only the values you are explicitly given.
> > > Anything else is simply a bug in your driver. 
> > 
> > It is unwise to assume that the hardware is in a sane state when the driver 
> > has been unloaded and reloaded. I agree that you should set the values that 
> > were explicitly given. That's why we should remember them.
> 
> No values are being explicitly given. Loading the driver should not cause
> any settings to be changed: changing the settings should do that!
> 
> There is no need for the drivers to change any settings. If the settings need
> to be (re)set, let userspace do it.
>  
> 
> James. 

Sure .. lets see you start a laptop in class or buisness meeting and have
everyone turn to look at you wondering why your laptop let off an ear
piercing shreak because the hardware defaults to all settings max.

And you will _STILL_ have that shriek for 1/2 - 1 second before the
userspace app loads.

And no we couldn't unplug either the mike or the speakers since they come
embedded in the laptop's case. 

I don't see in any of your trolling an answer for this problem.

	Gerhard

 

--
Gerhard Mack

gmack@innerfire.net

<>< As a computer I find your faith in technology amusing.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Horst von Brand]

"James A. Sutherland" <jas88@cam.ac.uk> said:
> On Mon, 06 Nov 2000, Horst von Brand wrote:

[...]

> > The problem (AFAIU) is that if the levels aren't set on startup, they are
> > random in some cases.

> So set them on startup. NOT when the driver is first loaded. Put it in
> the rc.d scripts.

There is a noticeable delay between to moment the module is insmod(8)ed,
and the moment when its settings are set by the startup script. Not funny
if it is going full blast ATM.
-- 
Horst von Brand                             vonbrand@sleipnir.valparaiso.cl
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Horst von Brand]

David Woodhouse <dwmw2@infradead.org> said:
> vonbrand@inf.utfsm.cl said:
> >  No funny "persistent data" mechanisms or screwups when the worker
> > gets removed and reinserted. In many cases the data module could be
> > shared among several others, in other cases it would have to be able
> > lo load several times or manage several incarnations of its payload. 

> The reason I brought this up now is because Keith's new 
> inter_module_register stuff could easily be used to provide this 
> functionality _without_ the extra module remaining loaded.

AFAIU, this is a acknowledged wart, to be added if there is no sane way out.
Better just loose it before it gets into the kernel, no?
-- 
Horst von Brand                             vonbrand@sleipnir.valparaiso.cl
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Horst von Brand]

David Woodhouse <dwmw2@infradead.org> said:

[...]

> No. You should initialise the hardware completely when the driver is 
> reloaded. Although the expected case is that the levels just happen to be 
> the same as the last time the module was loaded, you can't know that the 
> machine hasn't been suspended and resumed since then.

Oh? Suspending with the module loaded is forbidden then?
-- 
Horst von Brand                             vonbrand@sleipnir.valparaiso.cl
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Mon, 06 Nov 2000, Dan Hollis wrote:
> On Mon, 6 Nov 2000, James A. Sutherland wrote:
> > So autoload the module with a "dont_screw_with_mixer" option. When the kernel
> > first boots, initialise the mixer to suitable settings (load the module with 
> > "do_screw_with_mixer" or whatever); thereafter, the driver shouldn't change
> > the mixer settings on load.
> 
> You are asking for real trouble on hotplug hardware if you insist on this.

How so? There is no need for the driver to decide off its own bat to go
changing settings. If I plug in a hotplug soundcard and load the driver, I do
NOT want the driver to decide to set some settings. If I want settings set,
I'll do it myself (or have a script to do it).


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Mon, 06 Nov 2000, Evan Jeffrey wrote:
> > On Mon, 06 Nov 2000, David Woodhouse wrote:
> > > 
> > > No. You have to reset the hardware fully each time you load the module. 
> > > Although you _expect_ it to be in the state in which you left it, you can't
> >  
> > > be sure of that. 
> > 
> > If a reset is needed, I think it should come explicitly from userspace.
>  
> Take Alan's example of a USB device, say USB audio.  If it is disconnected
> and reconnected to add a hub, or anything else, the device may shut itself
> down, go to an undefined state, or even power cycle (if it draws power from 
> the USB +5V).  Same with hot-swap PCI cards.  The driver *MUST* reset the 
> device on load.  If saving mixer levels through this kind of transition is 
> desired (which it evidentally is), the module load/unload code must save and 
> restore the settings.

I'm not convinced.

> This is exactly equivelent to reseting hardware after a warm boot. 

Interesting. If that were done, my last sound card wouldn't have worked at all
under Linux: it had to be initialised under DOS before loading Linux. Had Linux
then reset everything, I'd have been soundless!

> Who knows
> what the Windows driver did to your card in the mean time. 

Either it initialises it (as it does, in this case), and I want (even NEED) it
to STAY initialised (i.e. if your driver does an unnecessary reset, your driver
is broken), or it doesn't, and I'll reset the card with an ioctl.

> A device driver
> can only guarantee that nobody horkes with its hardware while it is loaded--
> In the interim, the driver may have been connected to another computer,
> accessed by another driver, or accessed from userspace (say, VMWare doing
> a pass through driver).

So provide the FACILITY to reset the card, IFF it is needed. There are cases
where resetting the card is just stupid: don't force stupidity by design into
the kernel.

> I personally like the idea of having insmod/rmmod do copy-out/copy-in from
> a cache file in userspace.  That way, if we need large data sets (a ROM
> image for something.)  they don't take up kernel space when not in use.
> Also, it allows people to have persistant settings across reboot through
> the same mechanism--avoiding duplicating information in shutdown/startup 
> scripts.

I prefer the idea of the drivers which need this coming with a suitable
interface (/dev or /proc item) and a script to do this when needed.


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> changing settings. If I plug in a hotplug soundcard and load the driver, I do
> NOT want the driver to decide to set some settings. If I want settings set,
> I'll do it myself (or have a script to do it).

How about if your stuff is already nicely set up and you remove then replug
a device, or remove and swap for an identical replacement part. Most people then
want the configuration preserved.

And guess what the simple modutils solution using an ELF section solves that too
Want to go to default configuration ?

	rm /var/run/modules/eth0.data

or wrap it in a GUI.

[BTW windows gets the USB speaker state management right, seems they store all
the module persistent data in the registry!]


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Horst von Brand]

Oliver Xymoron <oxymoron@waste.org> say:

[...]

> Ioctl (or alternate device for plan9 groupies) is fine. My point is final
> initialization of the device is obviously delayed until the firmware is
> loaded.

Let's play perverse: What if I load the module, but don't initialize the
firmware, then unload? The fact that the firmware has been initialized has
to be kept somewhere...
-- 
Horst von Brand                             vonbrand@sleipnir.valparaiso.cl
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Mon, 06 Nov 2000, Gerhard Mack wrote:
> On Mon, 6 Nov 2000, James A. Sutherland wrote:
> 
> > Except this isn't possible with the hardware in question! If it were, there
> > would be no problem. In cases where the hardware doesn't support the
> > functionality userspace "needs", why put the kludge in the kernel?
> > 
> > If userspace wants to know what settings it set last time, it should store
> > those values somewhere.
> > 
> > > jas88@cam.ac.uk said:
> > > >  The right thing in this context is not to screw with hardware
> > > > settings unless and until it is given settings to set. Do not set
> > > > values arbitrarily: set only the values you are explicitly given.
> > > > Anything else is simply a bug in your driver. 
> > > 
> > > It is unwise to assume that the hardware is in a sane state when the driver 
> > > has been unloaded and reloaded. I agree that you should set the values that 
> > > were explicitly given. That's why we should remember them.
> > 
> > No values are being explicitly given. Loading the driver should not cause
> > any settings to be changed: changing the settings should do that!
> > 
> > There is no need for the drivers to change any settings. If the settings need
> > to be (re)set, let userspace do it.
> >  
> > 
> > James. 
> 
> Sure .. lets see you start a laptop in class or buisness meeting and have
> everyone turn to look at you wondering why your laptop let off an ear
> piercing shreak because the hardware defaults to all settings max.

That only happens if the driver is stupid enough to try guessing "correct"
volume settings. If it leaves well alone until it KNOWS the correct settings,
there is no ear piercing shriek. Nor is there any break in the sound if you
were listening to something from the mixer.

> And you will _STILL_ have that shriek for 1/2 - 1 second before the
> userspace app loads.

Wrong. The userspace app is the one triggering the device init, when it gives
the sound driver the correct volume settings. There is no half second delay.

> And no we couldn't unplug either the mike or the speakers since they come
> embedded in the laptop's case. 
> 
> I don't see in any of your trolling an answer for this problem.

It isn't trolling, and there is a perfectly simple answer, as I have already
explained.


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Tue, 07 Nov 2000, Alan Cox wrote:
> > changing settings. If I plug in a hotplug soundcard and load the driver, I do
> > NOT want the driver to decide to set some settings. If I want settings set,
> > I'll do it myself (or have a script to do it).
> 
> How about if your stuff is already nicely set up and you remove then replug
> a device,

When I plug it in and modprobe is triggered to load the driver, a script then
runs to feed the device appropriate configuration info. Since the driver only
resets the hardware when it is given the correct configuration, there's no
problem.

> or remove and swap for an identical replacement part. Most people then
> want the configuration preserved.

Hmm... define "identical". I take a laptop home, use a USB NIC to talk to my
LAN at home (using NAT) with a 192.168.* address. Then I take it elsewhere and
use the same model of NIC on the college LAN. All of a sudden, I get myself
banging on the door complaining about misconfigured NICs :-)
   
> And guess what the simple modutils solution using an ELF section solves that
> too Want to go to default configuration ?
> 
> 	rm /var/run/modules/eth0.data
> 
> or wrap it in a GUI.

That sounds a lot like what I've been advocating all along, if that file is
created/updated by a script when eth0's driver is unloaded, then fed back to
eth0 on load.

> [BTW windows gets the USB speaker state management right, seems they store all
> the module persistent data in the registry!]

Yes, that's what we need. A registry, so configuration problems can be
persistent across boots, and even reinstallations... ;-)


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Gerhard Mack]

On Tue, 7 Nov 2000, James A. Sutherland wrote:

> On Mon, 06 Nov 2000, Gerhard Mack wrote:
> > Sure .. lets see you start a laptop in class or buisness meeting and have
> > everyone turn to look at you wondering why your laptop let off an ear
> > piercing shreak because the hardware defaults to all settings max.
> 
> That only happens if the driver is stupid enough to try guessing "correct"
> volume settings. If it leaves well alone until it KNOWS the correct settings,
> there is no ear piercing shriek. Nor is there any break in the sound if you
> were listening to something from the mixer.
> 
> > And you will _STILL_ have that shriek for 1/2 - 1 second before the
> > userspace app loads.
> 
> Wrong. The userspace app is the one triggering the device init, when it gives
> the sound driver the correct volume settings. There is no half second delay.
> 
> > And no we couldn't unplug either the mike or the speakers since they come
> > embedded in the laptop's case. 
> > 
> > I don't see in any of your trolling an answer for this problem.
> 
> It isn't trolling, and there is a perfectly simple answer, as I have already
> explained.
> 

And if I don't use modules?

	Gerhard


--
Gerhard Mack

gmack@innerfire.net

<>< As a computer I find your faith in technology amusing.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Tue, 07 Nov 2000, Gerhard Mack wrote:
> On Tue, 7 Nov 2000, James A. Sutherland wrote:
> 
> > On Mon, 06 Nov 2000, Gerhard Mack wrote:
> > > Sure .. lets see you start a laptop in class or buisness meeting and have
> > > everyone turn to look at you wondering why your laptop let off an ear
> > > piercing shreak because the hardware defaults to all settings max.
> > 
> > That only happens if the driver is stupid enough to try guessing "correct"
> > volume settings. If it leaves well alone until it KNOWS the correct settings,
> > there is no ear piercing shriek. Nor is there any break in the sound if you
> > were listening to something from the mixer.
> > 
> > > And you will _STILL_ have that shriek for 1/2 - 1 second before the
> > > userspace app loads.
> > 
> > Wrong. The userspace app is the one triggering the device init, when it gives
> > the sound driver the correct volume settings. There is no half second delay.
> > 
> > > And no we couldn't unplug either the mike or the speakers since they come
> > > embedded in the laptop's case. 
> > > 
> > > I don't see in any of your trolling an answer for this problem.
> > 
> > It isn't trolling, and there is a perfectly simple answer, as I have already
> > explained.
> > 
> 
> And if I don't use modules?

Then none of this is relevant to you, since you can't unload any modules! And
now you're the one doing the trolling... WTF do you think module code is
supposed to do when you don't use modules?!


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Gerhard Mack]

> Then none of this is relevant to you, since you can't unload any modules! And
> now you're the one doing the trolling... WTF do you think module code is
> supposed to do when you don't use modules?!
> 

Simple ... I'd rather the hardware was set to 0 on startup but I know what
problems that presents to modules..

And no it wasn't the driver doing it afik. Sound card starts on max volume
as soon as it's initialised.

	Gerhard

--
Gerhard Mack

gmack@innerfire.net

<>< As a computer I find your faith in technology amusing.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Horst von Brand]

"James A. Sutherland" <jas88@cam.ac.uk> said:

[...]

> That only happens if the driver is stupid enough to try guessing "correct"
> volume settings.

It did not touch anything: By a fluke, or by default, the sound output was
going full blast, and the mike input was patched over to it ==> feedback
shriek until (a few tenths of second later) the volume settings _were_ set.
-- 
Horst von Brand                             vonbrand@sleipnir.valparaiso.cl
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Keith Owens]

On 06 Nov 2000 11:09:41 -0700, 
ebiederm@xmission.com (Eric W. Biederman) wrote:
>Well we don't have auto unload.

Check crontab, if it contains rmmod -a then you have auto unload.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Keith Owens]

On Mon, 6 Nov 2000 23:00:05 +0000 (GMT), 
Paul Jakma <paul@clubi.ie> wrote:
>On Mon, 6 Nov 2000, Alan Cox wrote:
>> Its called modules.conf. It has all these nice preload directives in it
>
>cool..
>
>doesn't seem to be documented though in modutils 2.3.17. what exactly
>does it do?

man modules.conf (yes, it _is_ documented).

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Persistent module storage - modutils design

[Keith Owens]

Enough people have asked for persistent module storage to at least
justify me writing the code.  The design is simple.

MODULE_PARM(var,type) currently defines type as [min[-max]]{b,h,i,l,s}.
For persistent data support, type is now [min[-max]]{b,h,i,l,s}{p}, the
trailing 'p' for persistent is optional.  Existing modutils only checks
one character after [min[-max]] so this is backwards compatible, no
need to upgrade modutils unless you want persistent data.

insmod takes parameters from modules.conf, from the saved persistent
data (see below) and from the command line, in that order.  The last
value for a parameter takes precedence.

rmmod locates the object for the module using the __insmod_xxx_O ksyms
entry.  If the object cannot be found or its timestamp has changed
since it was loaded then rmmod silently skips the persistent data.
Otherwise rmmod uses the .modinfo data in that object to determine the
address and type of the persistent parameters.  Each persistent
parameter is extracted from the module being unloaded, formatted as a
module parameter (e.g.  "irq=17") and written to /somewhere/module_name
which is a text file (vi is the ultimate configuration tool).

Almost all support is in user space.  The only kernel change is to add
'p' to the end of module parameters that are to be persistent.  Module
variables that are to be persistent and are not currently module
parameters need to be defined as MODULE_PARM().  The same kernel code
should work on 2.2 and 2.4 kernels, it should even work with modutils
2.1.121.

I have not decided where to save the persistent module parameters.  It
could be under /lib/modules/<version>/persist or it could be under
/var/log or /var/run.  I am tending towards /var/run/module_persist, in
any case it will be a modules.conf parameter.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage - modutils design

[Keith Owens]

On Tue, 07 Nov 2000 15:00:11 +1100, 
Keith Owens <kaos@ocs.com.au> wrote:
>insmod takes parameters from modules.conf, from the saved persistent
>data (see below) and from the command line, in that order.  The last
>value for a parameter takes precedence.

Correction: modprobe takes parameters from ...

insmod only does exactly what you tell it to.  It does not get parameter
values from modules.conf or anywhere else, only from the command line.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Tue, 07 Nov 2000, Gerhard Mack wrote:
> > Then none of this is relevant to you, since you can't unload any modules! And
> > now you're the one doing the trolling... WTF do you think module code is
> > supposed to do when you don't use modules?!
> > 
> 
> Simple ... I'd rather the hardware was set to 0 on startup but I know what
> problems that presents to modules..
> 
> And no it wasn't the driver doing it afik. Sound card starts on max volume
> as soon as it's initialised.

Which is why I want the driver to initialise the card to "known-good" settings.
Defaulting to 0 is not good enough.

With my approach, the driver is loaded as part of the kernel, but does NOT
initialise the card, it just confirms it is there. Then, during boot, a script
will initialise the sound card with the correct settings explicitly. That way,
the delay between card init and the card getting the correct settings is
minimal, even on broken hardware like this.


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Mon, 06 Nov 2000, Horst von Brand wrote:
> "James A. Sutherland" <jas88@cam.ac.uk> said:
> > On Mon, 06 Nov 2000, Horst von Brand wrote:
> 
> [...]
> 
> > > The problem (AFAIU) is that if the levels aren't set on startup, they are
> > > random in some cases.
> 
> > So set them on startup. NOT when the driver is first loaded. Put it in
> > the rc.d scripts.
> 
> There is a noticeable delay between to moment the module is insmod(8)ed,
> and the moment when its settings are set by the startup script. Not funny
> if it is going full blast ATM.

Yes, I know. That's why the driver MUST NOT change the volume settings when
insmod(8)ed, waiting instead until it gets specific settings from the script,
at which point it can initialise the card to the correct settings without a
delay.


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Helge Hafting]

Alan Cox wrote:
> 
> > It would probably be better (in this case) to increment the module count
> > when the mixer settings go above 0, and decrement it when the settings
> > go totally to 0.  This prevents an unwanted unload.
> 
> Thats about 200 lines of code and also about 50,000 emails complaining people
> cannot unload sound stuff.

Still, it is so logical.  Modules cannot be unloaded when in use.
"use" is usually "someone having the device open".  But
"someone listening to pass-through sound" is effectively using
the mixer - the module is in use in another way.  Network drivers
don't have a /dev/xxx to open/close either.

Having to turn off the master volume before unloading makes sense
to me.  (The driver may still free up DMA buffers when
nobody need them, i.e. when /dev/dsp haven't been used for some time.)


Helge Hafting
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> When I plug it in and modprobe is triggered to load the driver, a script then
> runs to feed the device appropriate configuration info. Since the driver only
> resets the hardware when it is given the correct configuration, there's no
> problem.

Thats another 100 lines of race prone network kernel code you dont need

> Hmm... define "identical". I take a laptop home, use a USB NIC to talk to my

Same Mac address or same serial number.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Tue, 07 Nov 2000, Alan Cox wrote:
> > When I plug it in and modprobe is triggered to load the driver, a script then
> > runs to feed the device appropriate configuration info. Since the driver only
> > resets the hardware when it is given the correct configuration, there's no
> > problem.
> 
> Thats another 100 lines of race prone network kernel code you dont need

Getting rid of 100 lines of code would certainly be worth doing...

Assuming I want the same configuration for the hardware as I did the last time
I used it is OK - provided that assumption is NOT in the kernel. As a default
behaviour in userspace, it's fine.

In the NIC example, I might well want the DHCP client to run whenever I
activate the card. Bringing the NIC up with the old configuration - which, with
dynamic IP addresses, could now include someone else's IP address! - is worse
than useless.

> > Hmm... define "identical". I take a laptop home, use a USB NIC to talk to my
> 
> Same Mac address or same serial number.

So if I take the NIC with me, Linux automagically misconfigures it for me. No
thanks.


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> In the NIC example, I might well want the DHCP client to run whenever I
> activate the card. Bringing the NIC up with the old configuration - which, with
> dynamic IP addresses, could now include someone else's IP address! - is worse
> than useless.

You'll notice the pcmcia subsystem already handles this, and keeps data in user
space although it doesnt support saving it back. And it all works

In your case it would be something like

eth0	pegasus
nopersist eth0
post-install eth0 /usr/local/sbin/my-dhcp-stuff

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage - modutils design

[Horst von Brand]

Keith Owens <kaos@ocs.com.au> said:

[...]

> I have not decided where to save the persistent module parameters.  It
> could be under /lib/modules/<version>/persist or it could be under
> /var/log or /var/run.  I am tending towards /var/run/module_persist, in
> any case it will be a modules.conf parameter.

/var/lib/persist/<version>/<wherever-the-module-is-in-/lib/modules/<version>/>

or some such. It has to match the kernel version somewhere (in case module
interfaces change), and it also should mirror the tree under
/lib/modules/<version> if for no other reason that there might show up
several modules named <foo>.
-- 
Dr. Horst H. von Brand                       mailto:vonbrand@inf.utfsm.cl
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[James A. Sutherland]

On Tue, 07 Nov 2000, Alan Cox wrote:
> > In the NIC example, I might well want the DHCP client to run whenever I
> > activate the card. Bringing the NIC up with the old configuration - which, with
> > dynamic IP addresses, could now include someone else's IP address! - is worse
> > than useless.
> 
> You'll notice the pcmcia subsystem already handles this, and keeps data in user
> space although it doesnt support saving it back. And it all works
> 
> In your case it would be something like
> 
> eth0	pegasus
> nopersist eth0
> post-install eth0 /usr/local/sbin/my-dhcp-stuff

So, in short, this is already done perfectly well in userspace without some
sort of Registry-style kernelside hack?


James.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Petko Manolov]

Alan Cox wrote:
> 
> > In the NIC example, I might well want the DHCP client to run whenever I
> > activate the card. Bringing the NIC up with the old configuration - which, with
> > dynamic IP addresses, could now include someone else's IP address! - is worse
> > than useless.
> 
> You'll notice the pcmcia subsystem already handles this, and keeps data in user
> space although it doesnt support saving it back. And it all works
> 
> In your case it would be something like
> 
> eth0    pegasus
> nopersist eth0
> post-install eth0 /usr/local/sbin/my-dhcp-stuff


Oops! Don't try to do this with pegasus.c older than 0.4.13.
I have set the ethernet address at open time, which breaks
dhcpd. I fixed that in test-10, but it's release took a long
time.

Sorry if the note doesn't make sense, i didn't follow the
thread from the beginning.


	Petkan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Alan Cox]

> > eth0	pegasus
> > nopersist eth0
> > post-install eth0 /usr/local/sbin/my-dhcp-stuff
> 
> So, in short, this is already done perfectly well in userspace without some
> sort of Registry-style kernelside hack?

Thats the idea. Once the rmmod code can read back values the cycle is complete
and the kernel doesnt care

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage - modutils design

[Horst von Brand]

Horst von Brand <vonbrand@inf.utfsm.cl> said:

[Yes, I know this is bad taste...]

> Keith Owens <kaos@ocs.com.au> said:
> 
> [...]
> 
> > I have not decided where to save the persistent module parameters.  It
> > could be under /lib/modules/<version>/persist or it could be under
> > /var/log or /var/run.  I am tending towards /var/run/module_persist, in
> > any case it will be a modules.conf parameter.
> 
> /var/lib/persist/<version>/<wherever-the-module-is-in-/lib/modules/<version>/>
> 
> or some such. It has to match the kernel version somewhere (in case module
> interfaces change), and it also should mirror the tree under
> /lib/modules/<version> if for no other reason that there might show up
> several modules named <foo>.

Note! This _has_ to be in the / filesystem so it works before mounting the
rest of the stuff (if ever). This would rule out /var, and leave just
/lib/modules/<version>. Makes me quite unhappy...
-- 
Dr. Horst H. von Brand                       mailto:vonbrand@inf.utfsm.cl
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage - modutils design

[Keith Owens]

On Tue, 07 Nov 2000 09:45:42 -0300, 
Horst von Brand <vonbrand@inf.utfsm.cl> wrote:
>Keith Owens <kaos@ocs.com.au> said:
>> I have not decided where to save the persistent module parameters.  It
>> could be under /lib/modules/<version>/persist or it could be under
>> /var/log or /var/run.  I am tending towards /var/run/module_persist, in
>> any case it will be a modules.conf parameter.
>
>/var/lib/persist/<version>/<wherever-the-module-is-in-/lib/modules/<version>/>
>
>or some such. It has to match the kernel version somewhere (in case module
>interfaces change),

But then you get the problem that upgrading from one kernel release to
the next loses the persistent data; either option has potential
problems.  I am tending towards no version number and have a flag on
insmod that says "the following parameters may not exist in the module,
be silent about any unknown symbols", the flag is automatically set by
modprobe for persistent parameters.

Going forward is not a problem, going backwards silently ignores
unknown parameters.

>and it also should mirror the tree under
>/lib/modules/<version> if for no other reason that there might show up
>several modules named <foo>.

It makes no sense to allow duplicate module names in the same kernel
tree.  "modprobe foo" - which one gets loaded?

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage - modutils design

[Keith Owens]

On Tue, 07 Nov 2000 10:30:39 -0300, 
Horst von Brand <vonbrand@inf.utfsm.cl> wrote:
>> Keith Owens <kaos@ocs.com.au> said:
>> > I have not decided where to save the persistent module parameters.  It
>> > could be under /lib/modules/<version>/persist or it could be under
>> > /var/log or /var/run.  I am tending towards /var/run/module_persist, in
>> > any case it will be a modules.conf parameter.
>
>Note! This _has_ to be in the / filesystem so it works before mounting the
>rest of the stuff (if ever). This would rule out /var, and leave just
>/lib/modules/<version>. Makes me quite unhappy...

Modules are loaded before non-root file systems are mounted, damn!
Looks like persistent data has to be stored in /lib/modules/persist (no
<version>, see earlier mail).  If somebody wants both a read only /lib
and persistent data then /lib/modules/persist must be a symlink and
they must mount the target file system before loading modules with
persistent data.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage - modutils design

[Alan Cox]

> Note! This _has_ to be in the / filesystem so it works before mounting the
> rest of the stuff (if ever). This would rule out /var, and leave just
> /lib/modules/<version>. Makes me quite unhappy...

The /lib filesystem is likely not writable so /var is the right default. 
Any reason it cant be overridden in modules.conf ?

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage - modutils design

[Horst von Brand]

Keith Owens <kaos@ocs.com.au> said:

[...]

> It makes no sense to allow duplicate module names in the same kernel
> tree.  "modprobe foo" - which one gets loaded?

Why the tree then?
-- 
Dr. Horst H. von Brand                       mailto:vonbrand@inf.utfsm.cl
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage - modutils design

[Keith Owens]

On Tue, 07 Nov 2000 11:47:57 -0300, 
Horst von Brand <vonbrand@inf.utfsm.cl> wrote:
>Keith Owens <kaos@ocs.com.au> said:
>> It makes no sense to allow duplicate module names in the same kernel
>> tree.  "modprobe foo" - which one gets loaded?
>
>Why the tree then?

Mainly so you can "modprobe -t net \*" and load all modules with /net/
in their pathname.  pcmcia modules need to be identified and linked.
mkinitrd also needs paths to identify scsi modules.  The old system of
assigning pathnames was manual, fragile and error prone, copying the
kernel tree works.  But whether modules are a flat directory, a
manually built two level directory or a copy of the kernel tree, it
still makes no sense to have duplicate names in the same kernel tree.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Jeff Garzik]

tytso@mit.edu wrote:
>    >      * Issue with notifiers that try to deregister themselves? (lnz;
>    >        notifier locking change by Garzik should backed out, according to
>    >        Jeff)
> 
>    and according to Alan
> 
> But it hasn't been backed out yet, correct?

It has been backed out.  Notifier register and deregister are locked,
but not notifier call.

-- 
Jeff Garzik             | "When I do this, my computer freezes."
Building 1024           |          -user
MandrakeSoft            | "Don't do that."
                        |          -level 1
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[Jeff Garzik]

tytso@mit.edu wrote:
> 
>    Date: Fri, 03 Nov 2000 16:10:50 -0500
>    From: Jeff Garzik <jgarzik@mandrakesoft.com>
> 
>    Part of that might be that serial doesn't support hotplug without
>    patching.
> 
> Did the patch which I sent out a few weeks ago actually work?  I haven't
> had time to get back to it, but I now have a cardbus card, so it's on my
> todo list....

I do not have CardBus card, so I can't test it.  Did you make sure to
have a pci_driver::remove function?  That is a requirement for PCI
hotplug.  Otherwise, your driver's probe routine will never be called.

I have a built-in Xircom modem on my Toshiba laptop, and it works great
with your patch.

	Jeff


-- 
Jeff Garzik             | "When I do this, my computer freezes."
Building 1024           |          -user
MandrakeSoft            | "Don't do that."
                        |          -level 1
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[tytso]

   Date: Fri, 3 Nov 2000 15:53:34 +0000 (GMT)
   From: Alan Cox <alan@lxorguk.ukuu.org.uk>

   >      * AIC7xxx doesnt work non PCI ? (Doug says OK, new version due
   >        anyway)

   This is now in Justin Gibbs hand but will take time to move on. Doug
   confirmed his current code is now merged too.

Does this mean that the problem has been fixed in the latest 2.4 tree?
i.e., Does Doug's current code have the problem fixed?

   >      * Issue with notifiers that try to deregister themselves? (lnz;
   >        notifier locking change by Garzik should backed out, according to
   >        Jeff)

   and according to Alan

But it hasn't been backed out yet, correct?  

Someone one send a patch to Linus?

						- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[tytso]

   Date: Fri, 03 Nov 2000 16:10:50 -0500
   From: Jeff Garzik <jgarzik@mandrakesoft.com>

   Part of that might be that serial doesn't support hotplug without
   patching.

Did the patch which I sent out a few weeks ago actually work?  I haven't
had time to get back to it, but I now have a cardbus card, so it's on my
todo list....

						- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[tytso]

   Date: Fri, 03 Nov 2000 17:20:53 -0500
   From: Jeff Garzik <jgarzik@mandrakesoft.com>

   > 4. Boot Time Failures
   > 
   >      * Crashes on boot on some Compaqs ? (may be fixed)

   compaq laptops?  desktops?  or alphas?

Absolutely no idea.  This was one I inherited from Alan's list.  If Alan
or someone else doesn't speak up on this one, I'll remove it from the
list....  (at a guess, it might be the docking station bug that's
already been fixed, but without more information I'm only guessing).

   >      * Fixing autofs4 to deal with VFS changes (Jeremy Fitzhardinge)

   I thought this was complete a long time ago?

Could be.  Jeremy?

					- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linux 2.4 Status / TODO page (Updated as of 2.4.0-test10)

[tytso]

   Date: Fri, 03 Nov 2000 18:32:31 -0800
   From: David Ford <david@linux.com>

   I reported pcmcia in all forms was broken for test8 on -my hardware-.

   Other kernels such as test10-prex that I'm on now are workable with dhinds
   pcmcia.  I sent you all the requested information you asked for in several
   forms.  The kernel's idea of the the sockets just doesn't work...again, on
   -my hardware-.

   It doesn't matter what voodoo you practice, all of the kernels in the last
   year have been unable to drive -my hardware- in any sort of stable fashion.
   Recent kernels just can't figure out IRQs for the sockets.

Ok, I've refined the PCMCIA comments in the buglist to be more specific.
It wasn't clear that anyone besides David Hinds was working on
stablizing the 2.4 kernel PCMCIA code, and Linus had already said that
he didn't consider it a show-stopper, and that he recommended people use
David's external PCMCIA/Cardbus package anyway.  So I assumed things had
continued to be in a completely broken state, and hadn't bothered to do
more digging into what was going on there.

						- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [RANT] Linux-IrDA status

[Michael Rothwell]

Linus Torvalds wrote:
> 
> On Tue, 7 Nov 2000, Michael Rothwell wrote:
> >
> > Linus, can you post reasons why you keep ignoring^W rejecting the IrDA
> > patch?
> 
> Basically, whatever Alan rants, I've not seen the patches all that many
> times at all.
> 
> Also, I've never seen much in the form of explanation, and at least the
> last patch I saw just the first screenful was so off-putting that I just
> went "Ok, I have real bugs to fix, I don't need this crap".
> 
>                 Linus


Like what? I'm not sure what you're saying here. It seems that the pople
writing the IrDA code have gotten no feedback from you as to why their
patch is never accepted -- could you clarify? They're apparently putting
a lot of effort into writing and fixing IrDA for Linux, and have become
very discouraged at the lack of feedback. "Crap" the code may be, but it
seems like it would be a good idea to at least say something substantive
about why their code keeps getting rejected.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [RANT] Linux-IrDA status

[Linus Torvalds]

On Wed, 8 Nov 2000, Michael Rothwell wrote:
> Linus Torvalds wrote:
> > 
> > Also, I've never seen much in the form of explanation, and at least the
> > last patch I saw just the first screenful was so off-putting that I just
> > went "Ok, I have real bugs to fix, I don't need this crap".
>
> Like what? I'm not sure what you're saying here. It seems that the pople
> writing the IrDA code have gotten no feedback from you as to why their
> patch is never accepted -- could you clarify?

There's one _major_ reason why things never get accepted:

 CVS trees

I'm not fed patches. I'm force-fed big changes every once in a while. I
don't like it.

I like it even less when the very first screen of a patch is basically a
stupid change that implies that somebody calls ioctl's from interrupts.

When I get a big patch like that, where the very first screen is
bletcherous, what the hell am I supposed to do? I'm not going to waste my
time on people who cannot send multiple small and well-defined patches,
and who send be big, ugly, "non-maintained" (as far as I'm concerned)
patches.

I'm surprised Alan rants about this. He knows VERY well how I work, and is
(along with Jeff Garzik and Randy Dunlap) one of the people who are very
good at sending me 25 separate patches with explanations of what they do.

Basically, if you send me a big patch with tons of changes, how the hell
DO you expect me to answer them? Does anybodt really expect me to go
through ten thousand lines of code that I do not know, and comment on it?
Obviously not, as anybody with an ounce of sense would see.

So what choice do I have? Apply them blindly?

Quite frankly, I'd rather have a few people hate me deeply than apply
stuff I don't like. If I just start blindly applying big patches, I can
avoid nasty discussions. But I'd rather have people flame me. Maybe some
day people will instead start sending me smaller commented patches.

I'm NOT going to do other peoples work for them. If people can't be
bothered to send me well-specified patches ESPECIALLY now that we're close
to 2.4.x, then I can't be bothered to apply them,

Live with it. Hat eme all you like. I do not care. Th ething I care about
is not letting too much crap through unchecked.

		Linus

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [RANT] Linux-IrDA status

[Linus Torvalds]

On Wed, 8 Nov 2000, Michael Rothwell wrote:
> 
> Like what? I'm not sure what you're saying here. It seems that the pople
> writing the IrDA code have gotten no feedback from you as to why their
> patch is never accepted -- could you clarify?

Just to clarify.

The ONLY message from the IrDA people I've gotten during the last few
weeks has been a SINGLE email from Dag Brattli, with a 330kB patch.

The whole, full, unabridged explanation for those 330kB of patches:

>> Hello Linus,
>> 
>> Here is the latest IrDA patch for Linux-2.4.0-test10. 
>> 
>> Short summary: 
>> 
>> o Fixes IrDA in 2.4
>> o Touches _no_ other files. 
>> 
>> Please apply! 
>> 
>> Best regards
>> 
>> Dag Brattli

That's it.

ONE message during the last month. ONE huge patch. From people who should
have known about 2.4.x being pending for some time. 

10,000+ lines of diff, with _no_ effort to split it up, or explain it with
anything but

	"o Fixes IrDA in 2.4"

and these people expect me to reply, sending long explanations of why I
don't like them? After they did nothing of the sort for the code they
claim should have been applied? Nada.

Get a grip. 

		Linus

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [RANT] Linux-IrDA status

[Russell King]

Linus Torvalds writes:
> ONE message during the last month. ONE huge patch. From people who should
> have known about 2.4.x being pending for some time. 
> 
> 10,000+ lines of diff, with _no_ effort to split it up, or explain it with
> anything but
> 
> 	"o Fixes IrDA in 2.4"
> 
> and these people expect me to reply, sending long explanations of why I
> don't like them? After they did nothing of the sort for the code they
> claim should have been applied? Nada.
> 
> Get a grip. 

Linus,

You know full well that I have sent you *small* self-contained obviously
correct patches since 2.4.0-test2 onwards.  Why haven't these been applied
when the only argument against it is "ONE huge patch"?
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |         Russell King        rmk@arm.linux.org.uk      --- ---
  | | | | http://www.arm.linux.org.uk/personal/aboutme.html   /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [RANT] Linux-IrDA status

[Dag Brattli]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 3611 bytes --]

Hi Linus,

I agree that the latest patch wasn't good about specifying its contents. 
But in fact, the 26th of august I sent you a mail which was much better
(but then your mailbox crashed or something!?) Since you hadn't applied 
any previoius patches (and not even the  patches from Russell), I felt that you
wasn't to interested about IrDA (even if Transmeta is a member of IrDA these 
days ;-) or didn't have time to look thru them. So that's the reason for the 
very short description. I'm sorry about that!

I've watched the ISDN discussion a year ago, so I already knew what you
felt about such large patches. The truth is that I've been very busy with my 
new job, and haven't had much time to maintain the Linux-IrDA project, so 
those large patches was the best I could do, and it's correct that I haven't  
actually flooded you with patches the last 6 months.

But we should anyway discuss what to do with IrDA support in Linux 2.4. 
The state of the current IrDA code in 2.4 is very bad and probably not 
working at all. The latest patch may have some bad code as well but at 
least things are working (and Linux isn't the OS which is best known for it's
beautiful code anyway). It will eventually be fixed, once people start 
complaining!

Some options:

1) Split up the large patch and fix the things you didn't like, submit them
with better discription. But then It's probably to late anyway for 2.4 (even if 
the 2.4-test series is not the most stable stuff I've tried). Is it to late for this?

2) Remove IrDA from the kernel, and we'll go back to using CVS and 
make our own package (like PCMCIA and IrDA was before they got 
into the kernel. At least PCMCIA used to work back then ;-)

3) Just apply the stuff!?! Look at Jean's mail for description of the changes.

-- Dag

On Tue, 7 Nov 2000 23:26:34 -0800 (PST), you wrote:
> 
> 
> On Wed, 8 Nov 2000, Michael Rothwell wrote:
> > 
> > Like what? I'm not sure what you're saying here. It seems that the pople
> > writing the IrDA code have gotten no feedback from you as to why their
> > patch is never accepted -- could you clarify?
> 
> Just to clarify.
> 
> The ONLY message from the IrDA people I've gotten during the last few
> weeks has been a SINGLE email from Dag Brattli, with a 330kB patch.
> 
> The whole, full, unabridged explanation for those 330kB of patches:
> 
> >> Hello Linus,
> >> 
> >> Here is the latest IrDA patch for Linux-2.4.0-test10. 
> >> 
> >> Short summary: 
> >> 
> >> o Fixes IrDA in 2.4
> >> o Touches _no_ other files. 
> >> 
> >> Please apply! 
> >> 
> >> Best regards
> >> 
> >> Dag Brattli
> 
> That's it.
> 
> ONE message during the last month. ONE huge patch. From people who should
> have known about 2.4.x being pending for some time. 
> 
> 10,000+ lines of diff, with _no_ effort to split it up, or explain it with
> anything but
> 
> 	"o Fixes IrDA in 2.4"
> 
> and these people expect me to reply, sending long explanations of why I
> don't like them? After they did nothing of the sort for the code they
> claim should have been applied? Nada.
> 
> Get a grip. 
> 
> 		Linus
> 
> 
> 
----
Dag Brattli,                       Mail:  dagb@fast.no
Senior Systems Engineer            Web:   http://www.fast.no/
Fast Search & Transfer ASA         Phone: +47 776 96 688
P.O. Box 1126                      Fax:   +47 776 96 689
NO-9261 Tromsø, NORWAY             Cell:  +47 924 05 388

Try FAST Search: http://www.alltheweb.com/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [RANT] Linux-IrDA status

[Michael Rothwell]

Linus Torvalds wrote:
> and these people expect me to reply, sending long explanations of why I
> don't like them? After they did nothing of the sort for the code they
> claim should have been applied? Nada.

Did you say that to them? I'm not saying you're wrong; but did you tell
them that? It might make your life easier if you make a faq on "how to
get your code accepted" and another on "how to get your code rejected."
Then you could send people off to read those, and maybe even site a
"violates #6" or whatever.

> Get a grip.

Help a little.

-M
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage [was Linux 2.4 Status / TODO page]

[Jamie Lokier]

Alan Cox wrote:
> Add a 'preserved' tag for one section of module memory. On load look up the
> data, if its from this boot memcpy it into the module. On unload write it
> back to disk. No kernel code needed.

I like!  No kernel code, yet no races or delay.

As written that removes the possibilities of variable length persistant
data, and the data is opaque to user space.

MODULE_PARM provides type information and structure to the data.  Why
not mark certain PARMS as persistent?  Not all would be named -- a block
of opaque data is useful.  But certain things like all the mixer levels
could be named parameters, giving you both persistant storage _and_
explicit configuration when you want that.  "s" PARMS (or similar)
can hold variable length data.

-- Jamie
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage - modutils design

[Rusty Russell]

In message <14032.973605093@ocs3.ocs-net> you write:
> On Tue, 07 Nov 2000 10:30:39 -0300, 
> Horst von Brand <vonbrand@inf.utfsm.cl> wrote:
> >Note! This _has_ to be in the / filesystem so it works before mounting the
> >rest of the stuff (if ever). This would rule out /var, and leave just
> >/lib/modules/<version>. Makes me quite unhappy...
> 
> Modules are loaded before non-root file systems are mounted, damn!

modules.conf already breaks FHS lib/ badly enough.  Modules loaded
before /var is mounted won't get persistant data.  Too bad; they
have to do something sane when it doesn't exist anyway.

> Looks like persistent data has to be stored in /lib/modules/persist (no
> <version>, see earlier mail).

You need versions: binary data is too prone to change (proven kernel
history).  It's the kernel installer's duty to know which ones can be
safely linked/copied to the new version.

Otherwise every data change requires a new symbol name: and this will
happen all the time.

Rusty.
--
Hacking time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage - modutils design

[H. Peter Anvin]

Followup to:  <20001109045247.BE39A8120@halfway.linuxcare.com.au>
By author:    Rusty Russell <rusty@linuxcare.com.au>
In newsgroup: linux.dev.kernel
> 
> > Modules are loaded before non-root file systems are mounted, damn!
> 
> modules.conf already breaks FHS lib/ badly enough.  Modules loaded
> before /var is mounted won't get persistant data.  Too bad; they
> have to do something sane when it doesn't exist anyway.
> 

Last I checked modules.conf was in /etc, not in /lib.

> 
> > Looks like persistent data has to be stored in /lib/modules/persist (no
> > <version>, see earlier mail).
> 
> You need versions: binary data is too prone to change (proven kernel
> history).  It's the kernel installer's duty to know which ones can be
> safely linked/copied to the new version.
> 
> Otherwise every data change requires a new symbol name: and this will
> happen all the time.
> 

Remember that we cannot rely on ANY form of persistent storage to be
available in the beginning; / may very well be readonly (on a ROM,
say.)  Since that means that we can't rely on writable storage being
available until at least one other filesystem has been mounted, it
might as well be the standard for variable data, i.e. /var.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage - modutils design

[Keith Owens]

On Thu, 09 Nov 2000 15:52:47 +1100, 
Rusty Russell <rusty@linuxcare.com.au> wrote:
>In message <14032.973605093@ocs3.ocs-net> you write:
>> Looks like persistent data has to be stored in /lib/modules/persist (no
>> <version>, see earlier mail).
>
>You need versions: binary data is too prone to change (proven kernel
>history).  It's the kernel installer's duty to know which ones can be
>safely linked/copied to the new version.

Not saving persistent data in binary format, so no problem.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage - modutils design

[Keith Owens]

On 8 Nov 2000 21:09:49 -0800, 
"H. Peter Anvin" <hpa@zytor.com> wrote:
>Remember that we cannot rely on ANY form of persistent storage to be
>available in the beginning; / may very well be readonly (on a ROM,
>say.)  Since that means that we can't rely on writable storage being
>available until at least one other filesystem has been mounted, it
>might as well be the standard for variable data, i.e. /var.

Agreed.  Default is /var/lib/modules-persist.  People who have a
separate /var partition that is mounted after modules are loaded can
use / and specify

persist /lib/modules/persist

in /etc/modules.conf.  If you have a separate /var and you do not want
to write to /, change your initialization scripts to load modules after
mounting /var.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Persistent module storage - modutils design

[Ralf Baechle]

On Tue, Nov 07, 2000 at 01:55:59PM +0000, Alan Cox wrote:

> > Note! This _has_ to be in the / filesystem so it works before mounting the
> > rest of the stuff (if ever). This would rule out /var, and leave just
> > /lib/modules/<version>. Makes me quite unhappy...
> 
> The /lib filesystem is likely not writable so /var is the right default. 

In theory yes.  In practice /var is often a mounted filesyste so for
modules loaded before /var is mounted this solution gets somewhat ugly.

  Ralf
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: The IrDA patches !!! (+ more flames)

[Jean Tourrilhes]

On Thu, Nov 09, 2000 at 07:24:04PM -0800, Jean Tourrilhes wrote:
> 
>         I spent my full day going through my archives and splitting
> the big patch of Dag into lots of small patches (see attached). I'm
> glad I've got a big hard drive full of junk.

	By the way, while I'm in flaming mode, could somebody tell ESR
that this patch split (as well as most of the patches themselves) was
sponsored by HP ? He should check his fact more carefully before
jumping on his guns, he seem one of the few who haven't visited the
Wireless LAN Howto...

	Now I can cool down...

	Jean
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

[Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Jeff V. Merkey]

[-- Attachment #1: Type: text/plain, Size: 895 bytes --]


The sendmail folks are claiming that the TCPIP stack in Linux is broken,
which is what they claim is causing problems on sendmail on Linux
platforms.  Before anyone says, "don't use that piece of shit sendmail,
use qmail instead", perhaps we should look at this problem and refute
these statements -- I think that sendmail is causing this problem.  The
version is sendmail 8.9.3

I can reproduce this bug on RH6.2, RH7.0, Caldera 2.2, 2.3, and 2.4,
Suse 6.X versions, and any of these distributions with the following
kernels.   

2.2.14, 2.2.16, 2.2.17, 2.2.18, 2.4.0 (all).  What happens is that
sendmail fails to forward mails with any attachments larger than 400K,
and they just sit in the /var/spool/mqueue directory for up to a week,
and eventually get delivered.

ANyone have any ideas if what the sendmail people are saying is on the
level, or is this just another sendmail bug.  

Jeff

[-- Attachment #2: Type: message/rfc822, Size: 7515 bytes --]

[-- Attachment #2.1.1: Type: text/plain, Size: 1538 bytes --]

Claus,

Here's the output from mailq -v and ps -ax.  As you can see, sendmail is
seriously malfunctining.  I have seen this same bug accross three
platforms with a wide variety 
of linux kernels from 2.2.14 to 2.2.18 to 2.4.0-10.  This is your bug,
and noone else's.

How do we get it fixed or is it possibly a configuration error.

Jeff

Claus Assmann wrote:
> 
> On Fri, Nov 10, 2000, Jeff V. Merkey wrote:
> >
> >
> > Claus Assmann wrote:
> > >
> > > On Fri, Nov 10, 2000, root wrote:
> > > >
> > > > I am seeimg sendmail 8.9.3 fail to deliver emails in /var/spool/mqueue
> > > > with attachements for up to a week.  Issuing the command "sendmail -v
> > > > -q" does not flush the mail queue.
> > >
> > > Is the queue item locked? What happens when you try to run it?
> >
> > How do I tell if the queue item is locked.  If you have a web browser, I
> 
> mailq -v
> look for a '*' next to the id.
> 
> > am running
> > webmin on the box and I have setup an account so you can come in and see
> > the mail queue yourself.
> >
> > enter www.timpanogas.org:10000 as your URL, then login as
> > username "sendmail" and password "sendmailbugs".   I will disable this
> 
> I don't know webmin, but most of the entries say "sending".  So
> check your sendmail processes to see what they are doing.  There
> might be some Linux TCP/IP stack bug that causes the problem (no
> proper timeout). For that you need to use trace/truss to see where
> the processes hang and how old they are.
> 
> I couldn't see any local delivery or very old entries.

[-- Attachment #2.1.2: ps.txt --]
[-- Type: text/plain, Size: 3116 bytes --]

  PID TTY      STAT   TIME COMMAND
    1 ?        S      0:06 init [3]
    2 ?        SW     0:00 [kflushd]
    3 ?        SW     0:01 [kupdate]
    4 ?        SW     0:00 [kpiod]
    5 ?        SW     0:00 [kswapd]
   95 ?        S      0:00 /sbin/modprobe -s -k nwfs
   96 ?        D      0:00 /sbin/modprobe -s -k nwfs
   97 ?        D      0:00 /sbin/modprobe -s -k nwfs
   98 ?        D      0:18 /sbin/modprobe -s -k nwfs
   99 ?        D      0:00 /sbin/modprobe -s -k nwfs
  100 ?        D      0:05 /sbin/modprobe -s -k nwfs
  101 ?        D      0:00 /sbin/modprobe -s -k nwfs
  102 ?        D      0:00 /sbin/modprobe -s -k nwfs
  103 ?        D      0:00 /sbin/modprobe -s -k nwfs
  104 ?        D      0:00 /sbin/modprobe -s -k nwfs
  105 ?        D      0:00 /sbin/modprobe -s -k nwfs
  344 ?        S      0:00 portmap
  367 ?        S      0:00 rpc.statd
  390 ?        S      0:00 /usr/sbin/automount /misc file /etc/auto.misc
  405 ?        S      0:00 /usr/sbin/apmd -p 10 -w 5 -W -s /etc/sysconfig/apm-sc
  456 ?        S      0:19 syslogd -m 0
  465 ?        S      0:00 klogd
  479 ?        S      0:00 identd -e -o
  483 ?        S      0:00 identd -e -o
  484 ?        S      0:00 identd -e -o
  485 ?        S      0:00 identd -e -o
  486 ?        S      0:00 identd -e -o
  497 ?        S      0:00 /usr/sbin/atd
  511 ?        S      0:00 crond
  529 ?        S      0:01 inetd
  543 ?        S      0:20 named -u named
  557 ?        S      0:00 lpd
  616 ?        S      0:00 gpm -t ps/2
  630 ?        S      0:00 httpd
  709 ?        S      0:00 xfs -droppriv -daemon -port -1
  750 ?        S      0:06 /usr/sbin/sshd
  755 tty2     S      0:00 /sbin/mingetty tty2
  756 tty3     S      0:00 /sbin/mingetty tty3
  757 tty4     S      0:00 /sbin/mingetty tty4
  758 tty5     S      0:00 /sbin/mingetty tty5
  759 tty6     S      0:00 /sbin/mingetty tty6
 2611 tty1     S      0:00 /sbin/mingetty tty1
16526 ?        S      0:00 httpd
16528 ?        S      0:00 httpd
18678 ?        S      0:00 httpd
18679 ?        S      0:00 httpd
18680 ?        S      0:00 httpd
18681 ?        S      0:00 httpd
18682 ?        S      0:00 httpd
18683 ?        S      0:00 httpd
19308 ?        S      0:00 httpd
19309 ?        S      0:00 httpd
19310 ?        S      0:00 httpd
20757 ?        S      0:00 httpd
20936 ?        S      0:00 httpd
20937 ?        S      0:00 httpd
20938 ?        S      0:00 httpd
20939 ?        S      0:00 httpd
20940 ?        S      0:00 httpd
20941 ?        S      0:00 httpd
22244 ?        S      0:00 sendmail: accepting connections on port 25
22331 ?        S      0:00 in.telnetd: manos.timpanogas.org                     
22332 pts/1    S      0:00 login -- jmerkey                    
22333 pts/1    S      0:00 -bash
22359 pts/1    S      0:00 su -
22360 pts/1    S      0:00 -bash
22638 ?        S      0:00 ftpd: cvx-36.flex.com: anonymous/guest@unknown.net: R
22841 ?        S      0:00 ftpd: cadmium.sge.net: anonymous/bpftp@: IDLE        
22866 ?        S      0:00 perl /usr/libexec/webmin/miniserv.pl /etc/webmin/mini
22875 pts/1    R      0:00 ps -ax

[-- Attachment #2.1.3: mailq.txt --]
[-- Type: text/plain, Size: 1679 bytes --]

		Mail Queue (11 requests)
--Q-ID-- --Size-- -Priority- ---Q-Time--- -----------Sender/Recipient-----------
FAA15716X   31418     200564 Nov  9 05:01 <linux-kernel-owner@vger.kernel.org>
          7BIT
					  <linux-archive@timpanogas.org>
					  <jmerkey@timpanogas.org>
FAA20318X   32693     201751 Nov 10 05:29 <linux-kernel-owner@vger.kernel.org>
          7BIT
					  <linux-archive@timpanogas.org>
					  <jmerkey@timpanogas.org>
SAA01998X   34484     203865 Nov  6 18:20 <linux-kernel-owner@vger.kernel.org>
          7BIT
					  <linux-archive@timpanogas.org>
					  <jmerkey@timpanogas.org>
QAA01341X   65091     204150 Nov  6 16:50 <linux-kernel-owner@vger.kernel.org>
          7BIT
					  <mharris@opensourceadvocate.org>
SAA13390X   41368     210478 Nov  8 18:03 <linux-kernel-owner@vger.kernel.org>
          7BIT
					  <linux-archive@timpanogas.org>
					  <jmerkey@timpanogas.org>
LAA03425X  158115     218595 Nov  6 11:27 <jmerkey@timpanogas.org>
					  <Mark.Coe@rrd.com>
					  <jmerkey@timpanogas.com>
QAA01343X   65091     234150 Nov  6 16:50 <linux-kernel-owner@vger.kernel.org>
          7BIT
					  <linux-archive@timpanogas.org>
					  <jmerkey@timpanogas.org>
KAA21225X  205041     235799 Nov 10 10:26 <paperboy@g2news.com>
      8BITMIME
					  <jmerkey@timpanogas.com>
FAA20229X    1457     272283+Nov 10 05:01 <mharris@opensourceadvocate.org>
                 (Warning: could not send message for past 1 hour)
					  <andre@linux-ide.org>
QAA06681X  242511     272929 Nov  7 16:18 <jmerkey@timpanogas.org>
      8BITMIME
					  <andre@timpanogas.org>
PAA12261X  576306     606701 Nov  8 15:06 <langus@timpanogas.com>
					  <jmerkey@timpanogas.com>

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Jeff V. Merkey]

"William F. Maton" wrote:
> 
> On Fri, 10 Nov 2000, Jeff V. Merkey wrote:
> 
> >
> > The sendmail folks are claiming that the TCPIP stack in Linux is broken,
> > which is what they claim is causing problems on sendmail on Linux
> > platforms.  Before anyone says, "don't use that piece of shit sendmail,
> > use qmail instead", perhaps we should look at this problem and refute
> > these statements -- I think that sendmail is causing this problem.  The
> > version is sendmail 8.9.3
> 
> What about sendmail 8.11.1?  Is the problem there too?

Yes.  Plus 8.11.1 has problems talking to older sendmails sine it uses
encryption.
> 
> >
> > I can reproduce this bug on RH6.2, RH7.0, Caldera 2.2, 2.3, and 2.4,
> > Suse 6.X versions, and any of these distributions with the following
> > kernels.
> >
> > 2.2.14, 2.2.16, 2.2.17, 2.2.18, 2.4.0 (all).  What happens is that
> > sendmail fails to forward mails with any attachments larger than 400K,
> > and they just sit in the /var/spool/mqueue directory for up to a week,
> > and eventually get delivered.
> >
> > ANyone have any ideas if what the sendmail people are saying is on the
> > level, or is this just another sendmail bug.
> >
> > Jeff
> 
> wfms
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[William F. Maton]

On Fri, 10 Nov 2000, Jeff V. Merkey wrote:

> 
> The sendmail folks are claiming that the TCPIP stack in Linux is broken,
> which is what they claim is causing problems on sendmail on Linux
> platforms.  Before anyone says, "don't use that piece of shit sendmail,
> use qmail instead", perhaps we should look at this problem and refute
> these statements -- I think that sendmail is causing this problem.  The
> version is sendmail 8.9.3

What about sendmail 8.11.1?  Is the problem there too?

> 
> I can reproduce this bug on RH6.2, RH7.0, Caldera 2.2, 2.3, and 2.4,
> Suse 6.X versions, and any of these distributions with the following
> kernels.   
> 
> 2.2.14, 2.2.16, 2.2.17, 2.2.18, 2.4.0 (all).  What happens is that
> sendmail fails to forward mails with any attachments larger than 400K,
> and they just sit in the /var/spool/mqueue directory for up to a week,
> and eventually get delivered.
> 
> ANyone have any ideas if what the sendmail people are saying is on the
> level, or is this just another sendmail bug.  
> 
> Jeff



wfms

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Jeff V. Merkey]

Richard A Nelson wrote:
> 
> Any `real` reason you're still at 8.9.3?   Current is 8.11.1
> 
> If you send me a note of the type that fails, (to cowboy@debian.org),
> it'll get received on both a 2.2.18-21/8.11.1 and 2.4.0-test10/8.11.2.Beta0

8.11.1 has problems talking to older sendmails and qmail.  I've seen
even worse problems on 8.11.1, and backreved it immediately, and the
encryption stuff has a lot of build problems on Linux. 

Jeff  


> system.
> 
> --
> Rick Nelson
> 'Mounten' wird für drei Dinge benutzt: 'Aufsitzen' auf Pferde, 'einklinken'
> von Festplatten in Dateisysteme, und, nun, 'besteigen' beim Sex.
>         -- Christa Keil
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Richard A Nelson]

Any `real` reason you're still at 8.9.3?   Current is 8.11.1

If you send me a note of the type that fails, (to cowboy@debian.org),
it'll get received on both a 2.2.18-21/8.11.1 and 2.4.0-test10/8.11.2.Beta0
system.

-- 
Rick Nelson
'Mounten' wird für drei Dinge benutzt: 'Aufsitzen' auf Pferde, 'einklinken'
von Festplatten in Dateisysteme, und, nun, 'besteigen' beim Sex.
	-- Christa Keil

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Jeff V. Merkey]

Horst von Brand wrote:
> 
> "Jeff V. Merkey" <jmerkey@timpanogas.org> SAID:
> > "William F. Maton" wrote:
> 
> [...]
> 
> > > What about sendmail 8.11.1?  Is the problem there too?
> 
> > Yes.  Plus 8.11.1 has problems talking to older sendmails sine it uses
> > encryption.
> 
> I've been using sendmail-8.11.1 (no encryption) to talk to MTAs all over
                                   ^^^^^^^^^^^^^^

Turn on encryption, and try sending attachements > 1MB and tell me if
you see any problems, like emails sitting in /var/spool/mqueue for a day
or two until they go out.  I can guarantee you will.

Jeff


> the place, many of them so old it is scary. No problems seen at this end.
> This is to be expected, BTW: They can't just go in and release an MTA which
> doesn't talk to the rest ot the world, now can they.
> --
> Dr. Horst H. von Brand                       mailto:vonbrand@inf.utfsm.cl
> Departamento de Informatica                     Fono: +56 32 654431
> Universidad Tecnica Federico Santa Maria              +56 32 654239
> Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Horst von Brand]

"Jeff V. Merkey" <jmerkey@timpanogas.org> SAID:
> "William F. Maton" wrote:

[...]

> > What about sendmail 8.11.1?  Is the problem there too?

> Yes.  Plus 8.11.1 has problems talking to older sendmails sine it uses
> encryption.

I've been using sendmail-8.11.1 (no encryption) to talk to MTAs all over
the place, many of them so old it is scary. No problems seen at this end.
This is to be expected, BTW: They can't just go in and release an MTA which
doesn't talk to the rest ot the world, now can they.
-- 
Dr. Horst H. von Brand                       mailto:vonbrand@inf.utfsm.cl
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Richard A Nelson]

On Fri, 10 Nov 2000, Jeff V. Merkey wrote:

> "William F. Maton" wrote:
> >
> > What about sendmail 8.11.1?  Is the problem there too?
>
> Yes.  Plus 8.11.1 has problems talking to older sendmails sine it uses
> encryption.

Eh?!? TLS is an optional, negotiated protocol started *after* the two
sendmails are communicating.

I've not seen any problems, save a documented case with *one* third
party SMTP server (don't recall who it was).

-- 
Rick Nelson
Old MacLinus had a stack/l-i-n-u-x/and on this stack he had a trace/l-i-n-u-x
with an Oops-Oops here and an Oops-Oops there
here an Oops, there an Oops, everywhere an Oops-Oops.
	-- tjimenez@site.gmu.edu, linux.dev.kernel

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Jeff V. Merkey]

Send me an email from it with an attachment > 1MB, and I will forward
back to you when (and if) It gets delivered before next week.

:-)

Jeff

Richard A Nelson wrote:
> 
> On Fri, 10 Nov 2000, Jeff V. Merkey wrote:
> 
> > "William F. Maton" wrote:
> > >
> > > What about sendmail 8.11.1?  Is the problem there too?
> >
> > Yes.  Plus 8.11.1 has problems talking to older sendmails sine it uses
> > encryption.
> 
> Eh?!? TLS is an optional, negotiated protocol started *after* the two
> sendmails are communicating.
> 
> I've not seen any problems, save a documented case with *one* third
> party SMTP server (don't recall who it was).
> 
> --
> Rick Nelson
> Old MacLinus had a stack/l-i-n-u-x/and on this stack he had a trace/l-i-n-u-x
> with an Oops-Oops here and an Oops-Oops there
> here an Oops, there an Oops, everywhere an Oops-Oops.
>         -- tjimenez@site.gmu.edu, linux.dev.kernel
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Richard A Nelson]

On Fri, 10 Nov 2000, Jeff V. Merkey wrote:

> 8.11.1 has problems talking to older sendmails and qmail.  I've seen
> even worse problems on 8.11.1, and backreved it immediately, and the
> encryption stuff has a lot of build problems on Linux.

Sounds like local build problems, possibly all the problems !

I can assist if you want to build 8.11.1 on Linux
-- 
Rick Nelson
Life'll kill ya                         -- Warren Zevon
Then you'll be dead                     -- Life'll kill ya

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Jeff V. Merkey]

Since I posted this on LKML, Claus over at sendmail.org seems more
motivated to track it down.  (since it might appear on the front page of
Linux today).  I would love your assistance Richard.  
It could be a local problem since smrsh also seems to be f_cked up as
well, but I am seeing the same thing with an out of the box RH6.2.

Jeff

Richard A Nelson wrote:
> 
> On Fri, 10 Nov 2000, Jeff V. Merkey wrote:
> 
> > 8.11.1 has problems talking to older sendmails and qmail.  I've seen
> > even worse problems on 8.11.1, and backreved it immediately, and the
> > encryption stuff has a lot of build problems on Linux.
> 
> Sounds like local build problems, possibly all the problems !
> 
> I can assist if you want to build 8.11.1 on Linux
> --
> Rick Nelson
> Life'll kill ya                         -- Warren Zevon
> Then you'll be dead                     -- Life'll kill ya
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[William F. Maton]

On Fri, 10 Nov 2000, Richard A Nelson wrote:

> On Fri, 10 Nov 2000, Jeff V. Merkey wrote:
> 
> > "William F. Maton" wrote:
> > >
> > > What about sendmail 8.11.1?  Is the problem there too?
> >
> > Yes.  Plus 8.11.1 has problems talking to older sendmails sine it uses
> > encryption.
> 
> Eh?!? TLS is an optional, negotiated protocol started *after* the two
> sendmails are communicating.

You anticipated what I was about to type :-)
 
> I've not seen any problems, save a documented case with *one* third
> party SMTP server (don't recall who it was).

No problems here either...

> 
> -- 
> Rick Nelson
> Old MacLinus had a stack/l-i-n-u-x/and on this stack he had a trace/l-i-n-u-x
> with an Oops-Oops here and an Oops-Oops there
> here an Oops, there an Oops, everywhere an Oops-Oops.
> 	-- tjimenez@site.gmu.edu, linux.dev.kernel
> 



wfms

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Jeff V. Merkey]

Claus is sloging into the box and we will be trying to track this down. 
If it is a problem in the Linux TCPIP stack, we'll post a report later
this afternoon as to where it looks like the problem is.  

Jeff

"Jeff V. Merkey" wrote:
> 
> Since I posted this on LKML, Claus over at sendmail.org seems more
> motivated to track it down.  (since it might appear on the front page of
> Linux today).  I would love your assistance Richard.
> It could be a local problem since smrsh also seems to be f_cked up as
> well, but I am seeing the same thing with an out of the box RH6.2.
> 
> Jeff
> 
> Richard A Nelson wrote:
> >
> > On Fri, 10 Nov 2000, Jeff V. Merkey wrote:
> >
> > > 8.11.1 has problems talking to older sendmails and qmail.  I've seen
> > > even worse problems on 8.11.1, and backreved it immediately, and the
> > > encryption stuff has a lot of build problems on Linux.
> >
> > Sounds like local build problems, possibly all the problems !
> >
> > I can assist if you want to build 8.11.1 on Linux
> > --
> > Rick Nelson
> > Life'll kill ya                         -- Warren Zevon
> > Then you'll be dead                     -- Life'll kill ya
> >
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@vger.kernel.org
> > Please read the FAQ at http://www.tux.org/lkml/
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Horst von Brand]

"Jeff V. Merkey" <jmerkey@timpanogas.org> said:
> Horst von Brand wrote:

[...]

> > I've been using sendmail-8.11.1 (no encryption) to talk to MTAs all over

> Turn on encryption, and try sending attachements > 1MB and tell me if
> you see any problems, like emails sitting in /var/spool/mqueue for a day
> or two until they go out.  I can guarantee you will.

No encryption use; and the maximal message size is 1Mb (for sanity's sake,
after somebody sent a PowerPoint presentation (some 3Mb), then thought that
perhaps the target didn't have PowerPoint, and sent it again with the
PowerPoint package, then noticed this might not work either, and sent it
_again_ with the full MS Office attached...  the joys of sysadminning ;-)
-- 
Dr. Horst H. von Brand                       mailto:vonbrand@inf.utfsm.cl
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Jeff V. Merkey]

"Richard B. Johnson" wrote:
> 
> On Fri, 10 Nov 2000, William F. Maton wrote:
> 
> > On Fri, 10 Nov 2000, Jeff V. Merkey wrote:
> >
> > >
> > > The sendmail folks are claiming that the TCPIP stack in Linux is broken,
> > > which is what they claim is causing problems on sendmail on Linux
> > > platforms.  Before anyone says, "don't use that piece of shit sendmail,
> > > use qmail instead", perhaps we should look at this problem and refute
> > > these statements -- I think that sendmail is causing this problem.  The
> > > version is sendmail 8.9.3
> >
> > What about sendmail 8.11.1?  Is the problem there too?
> >
> 
> I am running sendmail-8.11-0.Beta3 on Linux 2.4.0-test9. I didn't
> have any problem with it (except that the documentation sucks,
> making it extremely difficult to configure). Once configured, it runs
> fine. It also ran fine on Linux-2.2.17.
> 
> If something is staying in the mail-queue `mailq`, this means that
> the daemon isn't running. It may have crashed. This can be caused
> by somebody keeping some mailer entry in /etc/inetd.conf. Sendmail
> has to run as a daemon with no other interference on port 25.
> Check the configuration.

I did Dick.  The config is fine.  The daemon is also fine and running. 
What's really weird is that even if I do a "sendmail -v -q" command
(which should force the queue to flush) it still doesn't. 

Jeff

> 
> Cheers,
> Dick Johnson
> 
> Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).
> 
> "Memory is like gasoline. You use it up when you are running. Of
> course you get it all back when you reboot..."; Actual explanation
> obtained from the Micro$oft help desk.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Richard B. Johnson]

On Fri, 10 Nov 2000, William F. Maton wrote:

> On Fri, 10 Nov 2000, Jeff V. Merkey wrote:
> 
> > 
> > The sendmail folks are claiming that the TCPIP stack in Linux is broken,
> > which is what they claim is causing problems on sendmail on Linux
> > platforms.  Before anyone says, "don't use that piece of shit sendmail,
> > use qmail instead", perhaps we should look at this problem and refute
> > these statements -- I think that sendmail is causing this problem.  The
> > version is sendmail 8.9.3
> 
> What about sendmail 8.11.1?  Is the problem there too?
> 

I am running sendmail-8.11-0.Beta3 on Linux 2.4.0-test9. I didn't
have any problem with it (except that the documentation sucks,
making it extremely difficult to configure). Once configured, it runs
fine. It also ran fine on Linux-2.2.17.

If something is staying in the mail-queue `mailq`, this means that
the daemon isn't running. It may have crashed. This can be caused
by somebody keeping some mailer entry in /etc/inetd.conf. Sendmail
has to run as a daemon with no other interference on port 25.
Check the configuration.


Cheers,
Dick Johnson

Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Jeff V. Merkey]

Andrea,

All done.  It's already setup this way.

Jeff

Andrea Arcangeli wrote:
> 
> On Fri, Nov 10, 2000 at 11:45:39AM -0700, Jeff V. Merkey wrote:
> > > > > > [..]  Issuing the command "sendmail -v
> > > > > > -q" does not flush the mail queue. [..]
> 
> So first thing to do is to check that in /etc/sendmail.cf this line is
> commented out this way:
> 
> #O HostStatusDirectory=...
> 
> (if you build .cf via m4 add this line:
> 
> undefine(`confHOST_STATUS_DIRECTORY')dnl
> 
> and rebuild the .cf from the m4 source)
> 
> Then `rcsendmail reload; sendmail -q; mailq`.
> 
> Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Andrea Arcangeli]

On Fri, Nov 10, 2000 at 11:45:39AM -0700, Jeff V. Merkey wrote:
> > > > > [..]  Issuing the command "sendmail -v
> > > > > -q" does not flush the mail queue. [..]

So first thing to do is to check that in /etc/sendmail.cf this line is
commented out this way:

#O HostStatusDirectory=...

(if you build .cf via m4 add this line:

undefine(`confHOST_STATUS_DIRECTORY')dnl

and rebuild the .cf from the m4 source)

Then `rcsendmail reload; sendmail -q; mailq`.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Tim Walberg]

[-- Attachment #1: Type: text/plain, Size: 1036 bytes --]

On 11/10/2000 16:30 -0300, Horst von Brand wrote:
>>	"Jeff V. Merkey" <jmerkey@timpanogas.org> said:
>>	> Horst von Brand wrote:
>>	
>>	[...]
>>	
>>	> > I've been using sendmail-8.11.1 (no encryption) to talk to MTAs all over
>>	
>>	> Turn on encryption, and try sending attachements > 1MB and tell me if
>>	> you see any problems, like emails sitting in /var/spool/mqueue for a day
>>	> or two until they go out.  I can guarantee you will.
>>	
>>	No encryption use; and the maximal message size is 1Mb (for sanity's sake,
>>	after somebody sent a PowerPoint presentation (some 3Mb), then thought that
>>	perhaps the target didn't have PowerPoint, and sent it again with the
>>	PowerPoint package, then noticed this might not work either, and sent it
>>	_again_ with the full MS Office attached...  the joys of sysadminning ;-)
>>	-- 

Wow... that just might be one that's due for immortalizing
as an urban legend or what not... Definitely stupid user trick
material...


			tw


-- 
tewalberg@mediaone.net

[-- Attachment #2: Type: application/pgp-signature, Size: 175 bytes --]

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Andrea Arcangeli]

On Fri, Nov 10, 2000 at 12:34:40PM -0700, Jeff V. Merkey wrote:
> 
> Andrea,
> 
> All done.  It's already setup this way.

Ok. So please now show a tcpdump trace during the `sendmail -q` so we can see
what's going wrong in the TCP connection to the smtp server:

	tcpdump port smtp

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: The IrDA patches !!! (was Re: [RANT] Linux-IrDA status)

[Linus Torvalds]

On Thu, 9 Nov 2000, Jean Tourrilhes wrote:
> 
> 	I spent my full day going through my archives and splitting
> the big patch of Dag into lots of small patches (see attached). I'm
> glad I've got a big hard drive full of junk.

When I say multiple mails, I mean multiple mails. NOT "26 attachements in
one mail". In fact, not a single attachment at all, please. Send me
patches as a regular text body, with the explanation at the top, and the
patch just appended.

Why?

Attachements may look simple, but they are not. I end up having to open
each and every one of them individually, remembering which one I've
checked, save them off individually, remembering what the file name was,
and then apply them each individually.

See the picture? Attachements are evil. 

In contrast, imagine that you (and everybody else) sends me plain-text
patches, with just an explanation on top. What do I do?

I see the explanation immediately when I open the mail (ie when I press
the "n" key for "next email").

I can save it off with a simple "s../doit", which saves it in _one_ "doit"
file appended to all the other pending stuff. Alternatively, I can skip
it, or leave it pending, and let the _mail_software_ remember whether I
answered that particular patch.

I can reply to it individually, and that patch (and nothing else) will be
automatically set up for the reply so that I can easily quote whatever
parts I want to point out.

I can apply all the patches that I have approved with a single

	patch -p1 < ~/doit

without having to go through them individually.

None of the above works with attachments. 

> > Basically, if you send me a big patch with tons of changes, how the hell
> > DO you expect me to answer them? Does anybodt really expect me to go
> > through ten thousand lines of code that I do not know, and comment on it?
> > Obviously not, as anybody with an ounce of sense would see.
> 
> 	If somebody send you 1000 lines in one go or as 100 times 10
> lines, it doesn't matter, it is still 1000 lines of code to read
> through. Even small patches can be totally obscure for somebody not
> familiar with the code and what it is supposed to do.

You are WRONG.

10 emails with 1000-line patches are _much_ easier to handle. I can
clearly see the parts that belong together (nothing is mixed up with other
issues), and I can keep the explanation in mind. I do not have to remind
myself what that particular piece is doing.

It has other advantages too. With a single 10000-line patch, if I don't
like something, I have a hard time just removing THAT part. So I have to
reject the whole f*cking patch, and the person who sent it to me has to
fix up the whole thing (assuming I'd bother answering to it, poitning out
the parts that I don't like from the large patch, which I will not).

With 10 1000-line emails, I can decide to apply 8 of them outright, apply
one with comments, and discard one that does something particularly
nauseating. And I can much more easily explain to the submitter which one
I hate, without having to edit it down.

See?

		Linus

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Richard B. Johnson]

On Fri, 10 Nov 2000, Andrea Arcangeli wrote:

> On Fri, Nov 10, 2000 at 12:34:40PM -0700, Jeff V. Merkey wrote:
> > 
> > Andrea,
> > 
> > All done.  It's already setup this way.
> 
> Ok. So please now show a tcpdump trace during the `sendmail -q` so we can see
> what's going wrong in the TCP connection to the smtp server:
> 
> 	tcpdump port smtp
> 
> Andrea

I tried to send Jeff a 45 Megabyte file. It is still in the queue.


 FLAGS   UID   PID  PPID PRI  NI   SIZE   RSS WCHAN       STA TTY TIME COMMAND
[SNIPPED...]


   140     0    82     1   9   0    840   100 do_select   S   ?
   0:00 /usr/sbin/rpc.pcnfsd /var/spool/lpd 
   140     0    86     1   8   0   1744   364 do_select   S   ?
   0:00 sendmail: accepting connections 
    40     0  5742     1  16   0   1812   136 wait_for_tc S   ?   0:01
sendmail: ./eAAJm8V05731 vger.timpanogas.org.: client DATA 354 

It isn't a TCP/IP stack problem. It may be a memory problem. Every time
sendmail spawns a child to send the file data, it crashes.  That's
why the file never gets sent!

This is how /proc/meminfo looks right after it crashes. There has
been a lot of swapping going on.

        total:    used:    free:  shared: buffers:  cached:
Mem:  328114176 38932480 289181696        0  2293760 27115520
Swap: 139821056 10014720 129806336
MemTotal:       320424 kB
MemFree:        282404 kB
MemShared:           0 kB
Buffers:          2240 kB
Cached:          26484 kB
Active:           5576 kB
Inact_dirty:     18348 kB
Inact_clean:      4800 kB
Inact_target:      332 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       320424 kB
LowFree:        282400 kB
SwapTotal:      136544 kB
SwapFree:       126764 kB


Cheers,
Dick Johnson

Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Andrea Arcangeli]

On Fri, Nov 10, 2000 at 03:07:46PM -0500, Richard B. Johnson wrote:
> It isn't a TCP/IP stack problem. It may be a memory problem. Every time
> sendmail spawns a child to send the file data, it crashes.  That's
> why the file never gets sent!

Sure that could be the case. You should be able to verify the kernel kills the
task with `dmesg`.

However Jeff said the problem happens over 400K and a 500K attachment shouldn't
really run any machine out of memory, so maybe this wasn't his same problem?

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Jeff V. Merkey]

Andrea Arcangeli wrote:
> 
> On Fri, Nov 10, 2000 at 03:07:46PM -0500, Richard B. Johnson wrote:
> > It isn't a TCP/IP stack problem. It may be a memory problem. Every time
> > sendmail spawns a child to send the file data, it crashes.  That's
> > why the file never gets sent!
> 
> Sure that could be the case. You should be able to verify the kernel kills the
> task with `dmesg`.
> 
> However Jeff said the problem happens over 400K and a 500K attachment shouldn't
> really run any machine out of memory, so maybe this wasn't his same problem?

I think it is.  So it looks like sendmail is bombing when it attempts to
send large files. 

Jeff 

> 
> Andrea
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Jeff V. Merkey]

Andre,

SSH is running on this system, so send me your IP address to add to the
hosts.allow file and I'll send you an account so you can get into the
box and see just what's happening with ssh.  Andre Hedrick has root
privileges on this machine, so if I'm ever not around, he can get into
it.  I am running 2.2.18pre19 on i right now, and was going to upgrade
to 2.2.18-pre20 this evening, but let's look into this first since Alan
may have another patch to post ....
8)

Jeff



"Richard B. Johnson" wrote:
> 
> On Fri, 10 Nov 2000, Andrea Arcangeli wrote:
> 
> > On Fri, Nov 10, 2000 at 12:34:40PM -0700, Jeff V. Merkey wrote:
> > >
> > > Andrea,
> > >
> > > All done.  It's already setup this way.
> >
> > Ok. So please now show a tcpdump trace during the `sendmail -q` so we can see
> > what's going wrong in the TCP connection to the smtp server:
> >
> >       tcpdump port smtp
> >
> > Andrea
> 
> I tried to send Jeff a 45 Megabyte file. It is still in the queue.
> 
>  FLAGS   UID   PID  PPID PRI  NI   SIZE   RSS WCHAN       STA TTY TIME COMMAND
> [SNIPPED...]
> 
>    140     0    82     1   9   0    840   100 do_select   S   ?
>    0:00 /usr/sbin/rpc.pcnfsd /var/spool/lpd
>    140     0    86     1   8   0   1744   364 do_select   S   ?
>    0:00 sendmail: accepting connections
>     40     0  5742     1  16   0   1812   136 wait_for_tc S   ?   0:01
> sendmail: ./eAAJm8V05731 vger.timpanogas.org.: client DATA 354
> 
> It isn't a TCP/IP stack problem. It may be a memory problem. Every time
> sendmail spawns a child to send the file data, it crashes.  That's
> why the file never gets sent!
> 
> This is how /proc/meminfo looks right after it crashes. There has
> been a lot of swapping going on.
> 
>         total:    used:    free:  shared: buffers:  cached:
> Mem:  328114176 38932480 289181696        0  2293760 27115520
> Swap: 139821056 10014720 129806336
> MemTotal:       320424 kB
> MemFree:        282404 kB
> MemShared:           0 kB
> Buffers:          2240 kB
> Cached:          26484 kB
> Active:           5576 kB
> Inact_dirty:     18348 kB
> Inact_clean:      4800 kB
> Inact_target:      332 kB
> HighTotal:           0 kB
> HighFree:            0 kB
> LowTotal:       320424 kB
> LowFree:        282400 kB
> SwapTotal:      136544 kB
> SwapFree:       126764 kB
> 
> Cheers,
> Dick Johnson
> 
> Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).
> 
> "Memory is like gasoline. You use it up when you are running. Of
> course you get it all back when you reboot..."; Actual explanation
> obtained from the Micro$oft help desk.
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Richard B. Johnson]

On Fri, 10 Nov 2000, Jeff V. Merkey wrote:

> 
> 
> Andrea Arcangeli wrote:
> > 
> > On Fri, Nov 10, 2000 at 03:07:46PM -0500, Richard B. Johnson wrote:
> > > It isn't a TCP/IP stack problem. It may be a memory problem. Every time
> > > sendmail spawns a child to send the file data, it crashes.  That's
> > > why the file never gets sent!
> > 
> > Sure that could be the case. You should be able to verify the kernel kills the
> > task with `dmesg`.
> > 
> > However Jeff said the problem happens over 400K and a 500K attachment shouldn't
> > really run any machine out of memory, so maybe this wasn't his same problem?
> 
> I think it is.  So it looks like sendmail is bombing when it attempts to
> send large files. 
> 
> Jeff 
> 

Yes. I was finally able to send Jeff a very large file. I had
to get rid of all the other memory consumers on this system.

Once it had enough memory, it got sent.


Cheers,
Dick Johnson

Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Richard B. Johnson]

On Fri, 10 Nov 2000, Andrea Arcangeli wrote:

> On Fri, Nov 10, 2000 at 03:07:46PM -0500, Richard B. Johnson wrote:
> > It isn't a TCP/IP stack problem. It may be a memory problem. Every time
> > sendmail spawns a child to send the file data, it crashes.  That's
> > why the file never gets sent!
> 
> Sure that could be the case. You should be able to verify the kernel kills the
> task with `dmesg`.
> 
> However Jeff said the problem happens over 400K and a 500K attachment shouldn't
> really run any machine out of memory, so maybe this wasn't his same problem?
> 
> Andrea
> 

It ran out of memory. The file got sent fine after I got rid of
all the memory-consumers. Looks like a sendmail bug where they
expect to load a whole file into memory all at once before sending
it. I always thought you could read from a file, then write to
a socket. Maybe I'm old fashioned.


Cheers,
Dick Johnson

Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: sendmail fails to deliver mail with attachments in /var/spool/mqueue]

[Jeff V. Merkey]

"Richard B. Johnson" wrote:
> 
>
> 
> It ran out of memory. The file got sent fine after I got rid of
> all the memory-consumers. Looks like a sendmail bug where they
> expect to load a whole file into memory all at once before sending
> it. I always thought you could read from a file, then write to
> a socket. Maybe I'm old fashioned.
> 
> Cheers,
> Dick Johnson
>

Claus,

Looks like your bug.  As an FYI, sendmail.rpms in Suse, RedHat, and
OpenLinux all exhibit this behavior, which means they're all broken. 
Reading an entire file into memory must be a BSD feature.  I have
enabled an SSH account for you, so you can come in and debug.  Richard
also can get in and will be helping.

Jeff
-
To