Distributions vs kernel development

Distributions vs kernel development

[Stephen Hemminger]

After having being burned twice: first by Mandrake and supermount, and second
by SuSe and reiserfs attributes; are any of the distributions committed to
making sure that their distribution will run the standard kernel? (ie. 2.6.X from
kernel.org). When running a non-vendor kernel, I need to reasonably expect that the system
will boot and all the filesystems and standard devices are available.  I don't
expect every startup script to run clean, or every device that has a driver
only in the vendor kernel to work. 

But kernel developers need to be able run a standard environment. This effects
both day to day kernel testing and automated test environments like PLM and STP.
I am not saying it is bad that the distributions try to satisfy their customers,
or create a better experience; they just need to stop breaking things, and add
running a standard kernel as part of their QA cycle.

Re: Distributions vs kernel development

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 521 bytes --]

On Fri, 2004-05-07 at 17:53, Stephen Hemminger wrote:
> After having being burned twice: first by Mandrake and supermount, and second
> by SuSe and reiserfs attributes; are any of the distributions committed to
> making sure that their distribution will run the standard kernel? (ie. 2.6.X from
> kernel.org).

I can say that for Fedora Core we do have this goal. Obviously we do
expect a "recent enough" kernel, eg a 2.2 kernel on FC1 probably will
give issues, as will 2.4.0. 2.4.2x ought to be fine though.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Distributions vs kernel development

[Jeff Garzik]

Stephen Hemminger wrote:
> After having being burned twice: first by Mandrake and supermount, and second
> by SuSe and reiserfs attributes; are any of the distributions committed to
> making sure that their distribution will run the standard kernel? (ie. 2.6.X from
> kernel.org). When running a non-vendor kernel, I need to reasonably expect that the system
> will boot and all the filesystems and standard devices are available.  I don't
> expect every startup script to run clean, or every device that has a driver
> only in the vendor kernel to work. 

Fedora Core runs stock 2.4 and 2.6 kernels just fine...   :)

	Jeff

Re: Distributions vs kernel development

[Måns Rullgård]

Stephen Hemminger <shemminger@osdl.org> writes:

> After having being burned twice: first by Mandrake and supermount, and
> second by SuSe and reiserfs attributes; are any of the distributions
> committed to making sure that their distribution will run the standard
> kernel? (ie. 2.6.X from kernel.org). When running a non-vendor kernel,
> I need to reasonably expect that the system will boot and all the
> filesystems and standard devices are available.  I don't expect every
> startup script to run clean, or every device that has a driver only in
> the vendor kernel to work.

My last slackware install used an unmodified kernel, I don't know if
the latest versions so the same.  I run gentoo with vanilla 2.6
kernels.  I recently installed a vanilla 2.6 kernel on a redhat9
machine, without any problems (which quite surprised me).

-- 
Måns Rullgård
mru@kth.se

Re: Distributions vs kernel development

[Martin J. Bligh]

> After having being burned twice: first by Mandrake and supermount, and second
> by SuSe and reiserfs attributes; are any of the distributions committed to
> making sure that their distribution will run the standard kernel? (ie. 2.6.X from
> kernel.org). When running a non-vendor kernel, I need to reasonably expect that the system
> will boot and all the filesystems and standard devices are available.  I don't
> expect every startup script to run clean, or every device that has a driver
> only in the vendor kernel to work. 
> 
> But kernel developers need to be able run a standard environment. This effects
> both day to day kernel testing and automated test environments like PLM and STP.
> I am not saying it is bad that the distributions try to satisfy their customers,
> or create a better experience; they just need to stop breaking things, and add
> running a standard kernel as part of their QA cycle.

I run Debian to do this all the time. If you run the stable distro, it doens't
have all bleeding edge desktop crap, but it does have the major advantage of
actually working, and being perfectly stable. Getting it to run 2.6 needs
a few small updates, but nothing hard. 

Testing is very solid too, has 2.6 support out of the box, and more updated 
desktop stuff (I run that on my laptop). Unstable seems more stable than most
vendors shipped distros to me, but changes more rapidly. They also seem to
break serial console occasionally in unstable by loading fonts into it 
(idiots!), but that's trivial to fix.

The fact that their own kernel packages are so close to mainline probably 
helps a lot ;-)

M.

Re: Distributions vs kernel development

[Felipe Alfaro Solana]

On Fri, 2004-05-07 at 17:53, Stephen Hemminger wrote:
> After having being burned twice: first by Mandrake and supermount, and second
> by SuSe and reiserfs attributes; are any of the distributions committed to
> making sure that their distribution will run the standard kernel? (ie. 2.6.X from
> kernel.org). When running a non-vendor kernel, I need to reasonably expect that the system
> will boot and all the filesystems and standard devices are available.  I don't
> expect every startup script to run clean, or every device that has a driver
> only in the vendor kernel to work. 

Fedora Core 2 Test 3 runs beatifully with a stock kernel (I'm running
FC2T3 with a stock 2.6.6-rc3-mm2 kernel). I suggest you to take a look
at it.

Re: Distributions vs kernel development

[Timothy Miller]

Stephen Hemminger wrote:
> After having being burned twice: first by Mandrake and supermount, and second
> by SuSe and reiserfs attributes; are any of the distributions committed to
> making sure that their distribution will run the standard kernel? (ie. 2.6.X from
> kernel.org). 

I use Gentoo for this.

Re: Distributions vs kernel development

[Daniel Egger]

[-- Attachment #1: Type: text/plain, Size: 1155 bytes --]

On 07.05.2004, at 18:22, Martin J. Bligh wrote:

> Testing is very solid too, has 2.6 support out of the box, and more 
> updated
> desktop stuff (I run that on my laptop). Unstable seems more stable 
> than most
> vendors shipped distros to me, but changes more rapidly. They also 
> seem to
> break serial console occasionally in unstable by loading fonts into it
> (idiots!), but that's trivial to fix.

I'm running Debian stable, Debian stable/testing (mix) and Debian 
unstable
with both customized 2.4 and 2.6 as well as stock 2.4 and 2.6 kernels
without any sign of problems. Of course lm-sensors on top of a stock 2.4
kernel will not fly but everything else works quite fine.

I might also add that I can test quite a few combinations because many
of my machines are netbooting and thus picking up a different version
is a easy as relinking a directory and this always needs a selfcompiled
kernel because of the nfsroot and ip autoconfiguration settings. The
downside is that I needed to make slight modifications to the systems
to have partly separated temp dirs without the need for one complete
installation per machine.

Servus,
       Daniel

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 478 bytes --]

Re: Distributions vs kernel development

[Florian Weimer]

* Jeff Garzik:

> Fedora Core runs stock 2.4 and 2.6 kernels just fine...   :)

Have the problems with 2.4 kernels and Berkeley DB been fixed?

-- 
Current mail filters: many dial-up/DSL/cable modem hosts, and the
following domains: atlas.cz, bigpond.com, di-ve.com, hotmail.com,
jumpy.it, libero.it, netscape.net, postino.it, simplesnet.pt,
tiscali.co.uk, tiscali.cz, tiscali.it, voila.fr, yahoo.com.

Re: Distributions vs kernel development

[Paul Jakma]

On Fri, 7 May 2004, Florian Weimer wrote:

> * Jeff Garzik:
> 
> > Fedora Core runs stock 2.4 and 2.6 kernels just fine...   :)
> 
> Have the problems with 2.4 kernels and Berkeley DB been fixed?

And for i586, with either 2.4 nptl or 2.6, and BDB.

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
	warning: do not ever send email to spam@dishone.st
Fortune:
All new:
	Parts not interchangeable with previous model.

Re: Distributions vs kernel development

[Rene Rebe]

Hi,

On: Fri, 07 May 2004 12:03:00 -0400,
    Jeff Garzik <jgarzik@pobox.com> wrote:
> Stephen Hemminger wrote:
> > After having being burned twice: first by Mandrake and supermount, and second
> > by SuSe and reiserfs attributes; are any of the distributions committed to
> > making sure that their distribution will run the standard kernel? (ie. 2.6.X from
> > kernel.org). When running a non-vendor kernel, I need to reasonably expect that the system
> > will boot and all the filesystems and standard devices are available.  I don't
> > expect every startup script to run clean, or every device that has a driver
> > only in the vendor kernel to work. 
> 
> Fedora Core runs stock 2.4 and 2.6 kernels just fine...   :)
> 
> 	Jeff

Since other use this chance to propose already well known
distributions I just want to add that ROCK Linux is also designed to
run vanilla kernels - and in fact we only patch vitally important
changes (such as compile fixes / header fixes) into the -rock kernel.

  http://www.rocklinux.org

Sincerely yours,
  René Rebe
    - ROCK Linux stable release maintainer

--  
René Rebe - Europe/Germany/Berlin
  rene@rocklinux.org rene@rocklinux-consulting.de
http://www.rocklinux.org http://www.rocklinux-consulting.de

Re: Distributions vs kernel development

[John Bradford]

> Since other use this chance to propose already well known
> distributions

To be honest, why use a distribution at all, or if you do use one, why worry
about 'following' it after installation, instead of using it as a base from
which to do your own thing?

Of course, it sounds like a lot more effort to install whatever you need from
scratch, but you also more or less cut out distribution-specific issues, such
as one package depending on another, and of course you get much more
flexibility.

John.

Re: Distributions vs kernel development

[William Lee Irwin III]

On Sun, May 09, 2004 at 08:49:23AM +0200, Rene Rebe wrote:
> Since other use this chance to propose already well known
> distributions I just want to add that ROCK Linux is also designed to
> run vanilla kernels - and in fact we only patch vitally important
> changes (such as compile fixes / header fixes) into the -rock kernel.
>   http://www.rocklinux.org
> Sincerely yours,
>   Ren? Rebe
>     - ROCK Linux stable release maintainer

Very interesting. I've been quite eager to see a distro whose kernel
is based on mainline without significant adulteration. I'll have to get
a Rock Linux installation going and fiddle around with it.


-- wli

Re: Distributions vs kernel development

[Rene Rebe]

Hi,

On: Sun, 9 May 2004 08:07:06 +0100,
    John Bradford <john@grabjohn.com> wrote:
> > Since other use this chance to propose already well known
> > distributions
> 
> To be honest, why use a distribution at all, or if you do use one, why worry
> about 'following' it after installation, instead of using it as a base from
> which to do your own thing?

In this case you might want to know that ROCK Linux is not yet another
distribution but a distribution build kit including and build system
to rebuild the whole distribution in a sorted and clean manner.

And no, it is not like Gentoo where you need to rebuild on each box or
so.

> Of course, it sounds like a lot more effort to install whatever you need from
> scratch, but you also more or less cut out distribution-specific issues, such
> as one package depending on another, and of course you get much more
> flexibility.

Well, as I mentioned there is not much ROCK Linux specific in ROCK
Linux. Mostly it consists of this auto build system including a huge
set (1200 at the moment) package descriptions allowing convenient
single package or whole system (re-)builds.

  http://www.rocklinux.org/handbook.html

Sincerely yours,
  René Rebe
    - ROCK Linux stable release maintainer

--  
René Rebe - Europe/Germany/Berlin
  rene@rocklinux.org rene@rocklinux-consulting.de
http://www.rocklinux.org http://www.rocklinux-consulting.de

Re: Distributions vs kernel development

[Grzegorz Kulewski]

On Sun, 9 May 2004, Rene Rebe wrote:

> Hi,
> 
> On: Sun, 9 May 2004 08:07:06 +0100,
>     John Bradford <john@grabjohn.com> wrote:
> > > Since other use this chance to propose already well known
> > > distributions
> > 
> > To be honest, why use a distribution at all, or if you do use one, why worry
> > about 'following' it after installation, instead of using it as a base from
> > which to do your own thing?
> 
> In this case you might want to know that ROCK Linux is not yet another
> distribution but a distribution build kit including and build system
> to rebuild the whole distribution in a sorted and clean manner.
> 
> And no, it is not like Gentoo where you need to rebuild on each box or
> so.

You have binary packages in Gentoo so you can build once for many 
machines. All you need is to add one option to /etc/make.conf.


Grzegorz Kulewski

Re: Distributions vs kernel development

[Rene Rebe]

Hi,

On: Sun, 9 May 2004 10:59:48 +0200 (CEST),
    Grzegorz Kulewski <kangur@polcom.net> wrote:

> You have binary packages in Gentoo so you can build once for many 
> machines. All you need is to add one option to /etc/make.conf.

But the last time I took a look not even an installer or such. +
Gentoo has no support for custom modifications not even thinking about
a way to group such custom modifications / build configuration into a
well defined way to form a distribution. + ROCK Linux has a real
sandbox build environment, not this optimization via CFLAGS, and so on
Gentoo wannabe.

ROCK Linux is a build kit, allowing to define a set configurations /
modifications to create e.g. special embedded or server distributions
without the need to repackage all the packages and writing a build
script for those ... And when you read some ebuild scripts you find
the ROCK Linux automatics and tag based ASCI package description
format very pleasant, e.g.:

  http://svn2.rocklinux-consulting.de/rock-linux/trunk/package/base/rsync/

Sincerely yours,
  René Rebe
    - ROCK Linux stable release maintainer

--  
René Rebe - Europe/Germany/Berlin
  rene@rocklinux.org rene@rocklinux-consulting.de
http://www.rocklinux.org http://www.rocklinux-consulting.de

Re: Distributions vs kernel development

[John Bradford]

Quote from Rene Rebe <rene@rocklinux-consulting.de>:
> Hi,
> 
> On: Sun, 9 May 2004 08:07:06 +0100,
>     John Bradford <john@grabjohn.com> wrote:
> > > Since other use this chance to propose already well known
> > > distributions
> > 
> > To be honest, why use a distribution at all, or if you do use one, why worry
> > about 'following' it after installation, instead of using it as a base from
> > which to do your own thing?
> 
> In this case you might want to know that ROCK Linux is not yet another
> distribution but a distribution build kit including and build system
> to rebuild the whole distribution in a sorted and clean manner.

I must admit, I haven't had chance to have a proper look at ROCK Linux yet.

> And no, it is not like Gentoo where you need to rebuild on each box or
> so.

I keep hearing about projects which I hope will be what you describe above
for ROCK Linux.  Unfortunately, I haven't found one that goes far enough in
that direction yet.  Maybe there just isn't the demand for it these days.

I think that's unfortunate - installing the very basic parts of a system from
source, GLIBC, GCC, init scripts, system logging programs, yes, that is still
probably difficult for many new users, and a distribution that holds their
hand whilst doing that would be great.  However, once the basic system is
installed, many, many of the applications people want to run are very easily
installed, requiring little more than ./configure, make, and make install.

On paper, it might seem like an order of magnitude more effort, but in the
medium to long term, I think users would simply not encounter anywhere near
the number of problems they do by simply installing a distribution from CD.

Linux has caused a big wave in the computer industry, and brought free, open
source software to the attention of many people, but I think that even now,
much of the industry is, to a certain extent, trying to treat it in the same
way as proprietrary software.

Selling free, open source software as boxed products with support for that
boxed product seems to have worked up until now, but I think we'll quickly see
a plateau in interest in Linux, hopefully followed by a new wave of companies
working differently, with little emphasis on methods inherited from the
proprietrary software industry.

John.

Re: Distributions vs kernel development

[J. Ryan Earl]

Rene Rebe wrote:

>Hi,
>
>On: Sun, 9 May 2004 10:59:48 +0200 (CEST),
>    Grzegorz Kulewski <kangur@polcom.net> wrote:
>
>  
>
>>You have binary packages in Gentoo so you can build once for many 
>>machines. All you need is to add one option to /etc/make.conf.
>>    
>>
>
>But the last time I took a look not even an installer or such.
>
When was the last time you looked?  You mean no GUI, braindead installer? 

> +
>Gentoo has no support for custom modifications not even thinking about
>a way to group such custom modifications / build configuration into a
>well defined way to form a distribution. 
>
Wow, have you ever used Gentoo actually?  Gentoo is a Meta-Distribution: 
a distribution that describes itself.  Purely by -definition- you can 
modify it in any way you want.  There is no limit to how you can modify 
it, I'm not sure what gives you this idea.

Custom groups?  Group is such a vague term, but I'll give an example of 
a "custom group."  Say you have an identical set of 50 mcahines each 
with their own harddrives.  You can maintain your own snapshop of Gentoo 
(ie portage) on your own rsync server in which you have your own sandbox 
in which you test and validate updates before pushing them to the 
"group" of 50 machines in a prebuilt binary package format.

Just read this snippet from the sample make.conf:

# FEATURES are settings that affect the functionality of portage. Most of
#     these settings are for developer use, but some are available to non-
#     developers as well.
#
#  'autoaddcvs'  causes portage to automatically try to add files to cvs
#                that will have to be added later. Done at generation times
#                and only has an effect when 'cvs' is also set.
#  'buildpkg'    causes binary packages to be created of all packages that
#                are merged.
#  'ccache'      enables ccache support via CC.
#  'cvs'         feature for developers that causes portage to enable all
#                cvs features (commits, adds) and all USE flags in SRC_URI
#                will be applied for digests.
#  'digest'      autogenerate a digest for packages.
#  'distcc'      enables distcc support via CC.
#  'fixpackages' allows portage to fix binary packages that are stored in
#                PKGDIR. This can consume a lot of time. 'fixpackages' is
#                also a script that can be run at any given time to force
#                the same actions.
#  'keeptemp'    prevents the clean phase from deleting the temp files ($T)
#                from a merge.
#  'keepwork'    prevents the clean phase from deleting the WORKDIR.
#  'noauto'      causes ebuild to perform only the action requested and
#                not any other required actions like clean or
#  'noclean'     prevents portage from removing the source and temporary 
files
#                after a merge -- for debugging purposes only.
#  'nostrip'     prevents stripping of binaries.
#  'notitles'    disables xterm titlebar updates (which contain status 
info).
#  'sandbox'     enable sandbox-ing when running emerge and ebuild
#  'strict'      causes portage to react strongly to conditions that
#                have the potential to be dangerous -- like missing or
#                incorrect Manifest files.
#  'userpriv'    allows portage to drop root privleges while it is compiling
#                as a security measure, and as a side effect this can remove
#                sandbox access violations for users.
#  'usersandbox' enables sandboxing while portage is running under userpriv.
#                unpack -- for debugging purposes only.
#FEATURES="sandbox buildpkg ccache distcc userpriv usersandbox notitles 
noclean noauto cvs keeptemp keepwork"

>+ ROCK Linux has a real
>sandbox build environment, not this optimization via CFLAGS, and so on
>Gentoo wannabe.
>  
>
That's not right either, you can have your own sandbox (see above) in 
Gentoo in additional to custom CFLAGS and USE flags that customize which 
packages play with other packages.

>And when you read some ebuild scripts you find
>the ROCK Linux automatics and tag based ASCI package description
>format very pleasant, e.g.:
>
You can pimp ROCK without trying to knock Gentoo; Gentoo is quite 
powerful, well supported, and easy to extend customize.

-ryan

Re: Distributions vs kernel development

[J. Ryan Earl]

Timothy Miller wrote:

> Stephen Hemminger wrote:
>
>> After having being burned twice: first by Mandrake and supermount, 
>> and second
>> by SuSe and reiserfs attributes; are any of the distributions 
>> committed to
>> making sure that their distribution will run the standard kernel? 
>> (ie. 2.6.X from
>> kernel.org). 
>
>
> I use Gentoo for this.

As do I, I've never had a problem with a Vanilla kernel or one of 
Gentoo's maintained kernels.  Gentoo actually supports the 2.6 kernel, 
at least on AMD64 hardware that's all they support.  Though the vanilla 
kernels -do- work flawlessly, I still prefer the Gentoo patched kernels:

 >>> Unpacking linux-2.6.5.tar.bz2 to 
/var/tmp/portage/gentoo-dev-sources-2.6.5-r1/work
 * genpatches-2.6-5.29-base.tar.bz2 unpacked
 * genpatches-2.6-5.29-extras.tar.bz2 unpacked
 * Applying 
gentoo-dev-sources-2.6.5.CAN-2004-0109.patch...                                                    
[ ok ]
 * Applying 
1105_CAN-2004-0075-usb-vicam.patch...                                                              
[ ok ]
 * Applying 
1305_x86_64-2.6.5-rc3.patch...                                                                     
[ ok ]
 * Applying 
1310_k8_cardbus_io.patch...                                                                        
[ ok ]
 * Applying 
1315_alpha-sysctl-uac.patch...                                                                     
[ ok ]
 * Applying 
1905_bluetooth-oops.patch...                                                                       
[ ok ]
 * Applying 
2110_bcm5700_broadcom_gigabit_drvr_11272003.patch...                                               
[ ok ]
 * Applying 
2115_fa311-mac-address-fix.patch...                                                                
[ ok ]
 * Applying 
2320_adaptec_dpt_i2o.patch...                                                                      
[ ok ]
 * Applying 
2325_3ware-cmds_per_lun.patch...                                                                   
[ ok ]
 * Applying 
2705_powernow-k8-acpi.patch...                                                                     
[ ok ]
 * Applying 
3110_low-latency-cond_resched.patch...                                                             
[ ok ]
 * Applying 
3305_am9-2.6.4.patch...                                                                            
[ ok ]
 * Applying 
3310_cfq-4.patch...                                                                                
[ ok ]
 * Applying 
4105_lirc_infrared-2.6.5-rc2.patch...                                                              
[ ok ]
 * Applying 
4505_bootsplash-3.1.4-2.6.5-rc2.patch...                                                           
[ ok ]
 * Applying 
4705_squashfs-1.3r3.patch...                                                                       
[ ok ]
 * Applying 
4710_lufs-0.9.7-2.6.0-test9.patch...                                                               
[ ok ]
 * Applying 
4715_supermount-2.0.4-2.6.5_rc1.patch...                                                           
[ ok ]
 * Applying 
4720_gcloop-2.6-20040330.patch...                                                                  
[ ok ]
 * Applying 
4905_speakup_accessibility.patch...                                                                
[ ok ]
 >>> Source unpacked.

The k8/x86_64, broadcom, and bootsplash patches in particular make me 
happy.  They tend to stay within one or two minor kernel revisions of 
the current branch.  I've had production 2.6 servers for months.

-ryan

Re: Distributions vs kernel development

[Lech Szychowski]

> I've been quite eager to see a distro whose kernel
> is based on mainline without significant adulteration.

Take a look at Slackware, it uses the vanilla/Linus source.

-- 
	Leszek.

-- lech7@pse.pl 2:480/33.7          -- REAL programmers use INTEGERS --
-- speaking just for myself...

Re: Distributions vs kernel development

[Rob Landley]

On Sunday 09 May 2004 05:53, John Bradford wrote:

> > And no, it is not like Gentoo where you need to rebuild on each box or
> > so.
>
> I keep hearing about projects which I hope will be what you describe above
> for ROCK Linux.  Unfortunately, I haven't found one that goes far enough in
> that direction yet.  Maybe there just isn't the demand for it these days.

As a side effect of debugging busybox, I created a shellscript that compiles a 
stripped down Linux From Scratch system entirely from source.

By "debugging busybox" I mean upgrading it so that it can be used to replace 
the corresponding gnu packages (coreutils, patch, bash, diffutils, gawk, sed, 
bzip, findutils, grep, tar, util-linux...) and then an actual development 
environment (binutils, gcc, make, etc) built on top of that and the whole 
system successfully recompiled with itself.

I recently got it working to the point where you can chroot into the resulting 
system, and posted a version of the script to the busybox list a few days 
ago.  I have an upgraded version (based on Mariusz Mazur's 2.6.5.1 kernel 
headers and a lot of cleanups), and will be posting the script along with my 
current source tarball as soon as I figure out the best place to put a 134 
megabyte file that may be downloaded more than once.  (I need to attach a 
proper server to my cable modem, but this project is what I want to use, so 
there's a chicken and egg problem here... :)

I didn't know about Rock Linux until now.  I intend to look into it as soon as 
I clear my to-do list a bit, and maybe try to converge what I'm doing with 
that.

As for the rest of your article: don't assume that trading a familiar set of 
problems for an unfamiliar set of problems automatically solves more than it 
causes.  Build from source has more potential, sure.  And when the standard 
laptop has a 10 ghz 64 bit processor and 4 gigabytes of ram, it'll make a lot 
of sense (although compiling KDE or anything from the FSF will still suck).  
But compiling stuff from source is HEAVILY dependent on sequencing; 
./configure won't add ncurses support unless you installed ncurses first.  
How much stuff do you rebuild because you just added openssl, or switched to 
alsa from oss?  How do you keep track of the dependencies so it CAN rebuild?

If your build system is a static command set like my shellscript, "build this 
list of packages in this order", how much of an improvement is this really 
over a binary distro?  (Improvement, yes.  Panacea, no.)

Binary system bundlers protect people from a HUGE amount of crap.  Dragging 
this back on topic by the scruff of the neck, trying to get uclibc to work 
with a 2.6 kernel without having a 2.4 kernel on the box to loot headers from 
involves tracking down a third package just for the cleaned up headers.  Just 
knowing that you NEED to do that is black magic.  Grandma ain't dealing with 
this any time soon.  (Although mine have both taken to the internet 
remarkably well.  Both from windows machines, one via AOL...)

Rob

P.S.  I've looked at gentoo on several occasions.  Even downloaded the CD and 
tried to get up enthusiasm for inflicting it upon my laptop.  I've done Linux 
from scratch since version 3.0.  I've automated Linux From Scratch builds in 
various contexts for about four years now.  Installing Gentoo is just too 
much like work.

-- 
www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

Re: Distributions vs kernel development

[John Bradford]

Quote from Rob Landley <rob@landley.net>:
> On Sunday 09 May 2004 05:53, John Bradford wrote:
> 
> > > And no, it is not like Gentoo where you need to rebuild on each box or
> > > so.
> >
> > I keep hearing about projects which I hope will be what you describe above
> > for ROCK Linux.  Unfortunately, I haven't found one that goes far enough in
> > that direction yet.  Maybe there just isn't the demand for it these days.
> 
> As a side effect of debugging busybox, I created a shellscript that compiles a 
> stripped down Linux From Scratch system entirely from source.

That's interesting - what I find dissapointing about the Linux From Scratch
project is that it basically involves first setting up a minimal system, then
booting in to that new system at an early stage to complete the construction
of the system.

Although it lets the user 'escape' their original system quite quickly, my
interest lies more in creating a custom Linux installation using an existing
development environment.

[snip]

> As for the rest of your article: don't assume that trading a familiar set of 
> problems for an unfamiliar set of problems automatically solves more than it 
> causes.

Well, I don't just assume that, but in this case, I think I've demonstrated
to myself that pre-compiled binaries are more difficult to support than
programs compiled from source.

Of course, it's possible to learn a lot about one specific version of one
specific distribution, and doing that might make it easier to support any
pre-compiled binaries in that particular version of that particular
distribution than an installation from source.

However, surely that is just learning about more and more special cases.

It might even be possible to learn about such a large number of special cases
that a user can solve all the problems they encounter without any generic
knowledge of a system, but I think that is a bad thing to encourage.

If I get a phone call asking for help with a program I've never heard of
before, possibly on a distribution I've never heard of before, I belive that
I can offer much better generic advice based on an overall knowledge of the
system if everything is source-based, than if a distribution's pre-compiled
binaries have been installed.  To me, those pre-compiled binaries are
"non-standard".

This is one reason why I do not advertise support services for proprietrary
systems.  I do not want to fix problems by learning about many, many, special
cases - I want to use my generic knowledge of computers to fix problems, and
I believe that it is a better way to use computers in general.

> Build from source has more potential, sure.  And when the standard 
> laptop has a 10 ghz 64 bit processor and 4 gigabytes of ram, it'll make a lot 
> of sense

I don't think compiling from source is particularly inpractical on a typical
modern desktop machine today.

[snip]

> But compiling stuff from source is HEAVILY dependent on sequencing; 
> ./configure won't add ncurses support unless you installed ncurses first.  

That's true for a lot of libraries and basic components of a system, but I
don't think it's so much of a problem in general.

> How much stuff do you rebuild because you just added openssl, or switched to 
> alsa from oss?

Not too much to make it impractical, in my opinion.

> How do you keep track of the dependencies so it CAN rebuild?

In practice, I haven't found it to be particularly complicated.

> If your build system is a static command set like my shellscript, "build this 
> list of packages in this order",

It's not.  My 'build system' is basically me sitting at the console.

> how much of an improvement is this really 
> over a binary distro?

For the initial installation, maybe not much of an improvement, but for
on-going use of a system, I think compiling from source is better overall.

John.

Re: Distributions vs kernel development

[Rob Landley]

On Wednesday 19 May 2004 03:49, John Bradford wrote:
> Quote from Rob Landley <rob@landley.net>:
> > On Sunday 09 May 2004 05:53, John Bradford wrote:
> > > > And no, it is not like Gentoo where you need to rebuild on each box
> > > > or so.
> > >
> > > I keep hearing about projects which I hope will be what you describe
> > > above for ROCK Linux.  Unfortunately, I haven't found one that goes far
> > > enough in that direction yet.  Maybe there just isn't the demand for it
> > > these days.
> >
> > As a side effect of debugging busybox, I created a shellscript that
> > compiles a stripped down Linux From Scratch system entirely from source.
>
> That's interesting - what I find dissapointing about the Linux From Scratch
> project is that it basically involves first setting up a minimal system,
> then booting in to that new system at an early stage to complete the
> construction of the system.

It's not really a distro.  It's an enormous HOWTO.  (Then again, so's Gentoo, 
but gentoo does claim to be a distro, which is where I get disappointed...)

> Although it lets the user 'escape' their original system quite quickly, my
> interest lies more in creating a custom Linux installation using an
> existing development environment.

Earlier versions of Linux From Scratch did that; they didn't have the 
intermediate system, you just did a build from the main system to get a 
result.  Unfortunately, it would do things like break strangely on certain 
versions of debian due to environment variables or the way they'd patched 
make, and they had to have various warnings and "if it does this, do this..."

The two stage thing of minimal build, chroot, build final system made it a lot 
easier to reproduce regardless of the starting distro.

> [snip]
>
> > As for the rest of your article: don't assume that trading a familiar set
> > of problems for an unfamiliar set of problems automatically solves more
> > than it causes.
>
> Well, I don't just assume that, but in this case, I think I've demonstrated
> to myself that pre-compiled binaries are more difficult to support than
> programs compiled from source.

And how many users are you trying to support?  (This varies depending on scale 
of deployment.  Been there, done that...)

Being _able_ to rebuild the system with printf's in it is a Very Good Thing.  
But presumably, that's what Red Hat's SRPMs are for...

> Of course, it's possible to learn a lot about one specific version of one
> specific distribution, and doing that might make it easier to support any
> pre-compiled binaries in that particular version of that particular
> distribution than an installation from source.

Your support idiosyncrasies are going to differ from GCC 3.3.1 vs GCC 3.3.3, 
let alone 3.3 vs 3.4 or 3.5.  And of course if they compiled it from source, 
but did so with the funky broken gcc 2.9.6 in Red Hat 8 or whatever it was, 
then you could still have strange bugs that have nothing to do with the 
actual package.

As I said, you're really just exchanging one set of problems for another.  
It's not a magic bullet.

> However, surely that is just learning about more and more special cases.

All tech support is learning about more and more special cases.  You learn how 
it _should_ work (which is often something you don't start out knowing, 
because when it's working you don't HAVE to know).  You learn specific ways 
it can break (by encountering them).  You learn to ask the right questions 
when the user did something totally unrelated that broke it and doesn't think 
to volunteer this information because it _IS_ seemingly totally unrelated 
(like upgrading the kernel or libc, switching on ECN, moving the sucker 
behind a masquerading router...)

> It might even be possible to learn about such a large number of special
> cases that a user can solve all the problems they encounter without any
> generic knowledge of a system, but I think that is a bad thing to
> encourage.

I.E. if the user becomes enough of an expert they don't need you to support 
them anymore.  While theoretically possible, this is not actually a useful 
observation in the field much.

> If I get a phone call asking for help with a program I've never heard of
> before, possibly on a distribution I've never heard of before, I belive
> that I can offer much better generic advice based on an overall knowledge
> of the system if everything is source-based, than if a distribution's
> pre-compiled binaries have been installed.  To me, those pre-compiled
> binaries are "non-standard".

Red Hat _is_ source based.  So is Debian.  The fact they build it on their 
servers rather than the target machine doesn't mean they don't give you the 
source to build it yourself if you want to.  However, figuring out how to 
reproduce their build process can be a flaming pain.

The same is true with build-on-the-target-system distributions.  I put 
together one based on a bash script, which is as simple as I could make it 
and designed to have the bits cut and pasted out if you need to rebuild 
stuff, but you've still got to remember to set the right environment 
variables for source and target directories.

The uclibc project has "buildroot", where everything is done by a giant 
makefile.  I hate makefiles: languages should be either declarative or 
imparative, you should not mix both in the same environment.  Also, it has a 
bunch of rough edges like the fact I once typed "make clean" and it deleted 
stuff out of my parent system.  I could presumably figure out what a given 
segment of that is doing and boil it down to the appropraite ./configure make 
make install commands (and the patches and such they apply, and the symlinks 
and config files they create, and...)  But it wouldn't exactly be a 60 second 
job.

And then there's gentoo, which has a python script talk to a server out on the 
net to figure out what to build.  If you're going to even TRIGGER that, you 
need to be familiar with their packaging tool.  To take it apart and modify 
the build would take a lot of eyeballing.

That said, I once spent four hours diagnosing the fact that a friend's Windows 
98 machine was booting into safe mode because they'd deleted a font that was 
used by one of the more obscure help files.  (Because the help file used that 
font, that meant help system referenced this font while indexing files on 
bootup.  And when it didn't find it, it aborted the boot.  Thus it was 
booting into "safe mode", a diagnostic mode where none of the diagnostic 
tools actually run.  Brilliant.  I copied another font to that name and told 
them not to do it again.)

> This is one reason why I do not advertise support services for proprietrary
> systems.  I do not want to fix problems by learning about many, many,
> special cases - I want to use my generic knowledge of computers to fix
> problems, and I believe that it is a better way to use computers in
> general.

I.E. you don't want to do tech support.

Even with all the source, tech support is STILL knowing about many many 
special cases.  Yes, you can work it out from first principles, and having 
the source code in a readily buildable state where you can compile and drop 
in functional replacements (this is NOT the same thing as just having the 
source code) can help.

But if you're working out the answer from first principles every time you 
provide somebody with tech support, you're going to be an amazingly 
inefficient support person.

> > Build from source has more potential, sure.  And when the standard
> > laptop has a 10 ghz 64 bit processor and 4 gigabytes of ram, it'll make a
> > lot of sense
>
> I don't think compiling from source is particularly inpractical on a
> typical modern desktop machine today.

You've never built kde then.

> [snip]
>
> > But compiling stuff from source is HEAVILY dependent on sequencing;
> > ./configure won't add ncurses support unless you installed ncurses first.
>
> That's true for a lot of libraries and basic components of a system, but I
> don't think it's so much of a problem in general.

You've tried it, then?

> > How much stuff do you rebuild because you just added openssl, or switched
> > to alsa from oss?
>
> Not too much to make it impractical, in my opinion.

And if the end-user decides to upgrade their system with stuff it didn't have 
before?

This isn't restricted to source distributions, by the way.  I tried to update 
a friend's SuSE system to the 2.6 kernel.  I expected to have to upgrade a 
number of packages, but what I didn't expect was that SuSE doesn't seem to 
have have ncurses installed (or at least not something make menuconfig can 
find).  Python in Red Hat didn't have ncurses support last I checked, either.  
(RH 9, I think...)

Suppose they don't select OpenSSL because they go "this is a desktop system, 
not a server", and then they realise later "oh, I need https:// support in 
Konqueror/Mozilla"...

> > How do you keep track of the dependencies so it CAN rebuild?
>
> In practice, I haven't found it to be particularly complicated.

How much practice have you found it in?

> > If your build system is a static command set like my shellscript, "build
> > this list of packages in this order",
>
> It's not.  My 'build system' is basically me sitting at the console.

I've supported end-users before.  I've built systems for people who can't roll 
their own.  Lots of the problems they have are non-obvious from the 
perspective of geekdom.

> > how much of an improvement is this really
> > over a binary distro?
>
> For the initial installation, maybe not much of an improvement, but for
> on-going use of a system, I think compiling from source is better overall.

Name me a Linux distribution that is not "compiled from source".

You keep saying that installing from source is better, but it seems to be from 
"gee, wouldn't it be nice if" land rather than due to actual experience.  You 
_can_ build and install an SRPM into a Red Hat system.  It's too much of a 
pain to be worth it to me, but it's been an option for years and years.

Most desktop users I've seen who modify their own systems dirty them badly 
enough that they copy their data off and reinstall every year or so anyway, 
so your long term support argument is a bit strange.  (We make fun of this as 
a windows thing, but in the LInux world people DO reinstall when the next Red 
Hat comes out...)  And having the server setup reproducible from a fresh 
install is just plain good hygiene, anyway.  (Doesn't mean everybody does it, 
but carrying around verbatim binary images through three or four hardware 
upgrades is _not_ a recipe for maintainability.  And what do you do if the 
sucker gets rootkitted, anyway?  Hope you got it all cleaned out?)

I'm not saying compiling from source is BAD.  I'm just saying it's not as big 
an improvement as you seem to think it will be.  It does allow a number of 
new things to be done, but by itself it doesn't make a major difference.

> John.

Rob

-- 
www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

Re: Distributions vs kernel development

[John Bradford]

Hi,

Quote from Rob Landley <rob@landley.net>:
> On Wednesday 19 May 2004 03:49, John Bradford wrote:
> > Quote from Rob Landley <rob@landley.net>:
> > > On Sunday 09 May 2004 05:53, John Bradford wrote:
> > > > > And no, it is not like Gentoo where you need to rebuild on each box
> > > > > or so.
> > > >
> > > > I keep hearing about projects which I hope will be what you describe
> > > > above for ROCK Linux.  Unfortunately, I haven't found one that goes far
> > > > enough in that direction yet.  Maybe there just isn't the demand for it
> > > > these days.
> > >
> > > As a side effect of debugging busybox, I created a shellscript that
> > > compiles a stripped down Linux From Scratch system entirely from source.
> >
> > That's interesting - what I find dissapointing about the Linux From Scratch
> > project is that it basically involves first setting up a minimal system,
> > then booting in to that new system at an early stage to complete the
> > construction of the system.
> 
> It's not really a distro.  It's an enormous HOWTO.  (Then again, so's Gentoo, 
> but gentoo does claim to be a distro, which is where I get disappointed...)

I disagree - they supply their own init scripts, for example, rather than
guiding people towards writing their own.  That is not a criticism of Linux
>From Scratch, just an observation.  Ultimately, users are not putting 100%
of their final system together from scratch.

However, in practice you are right it is 99% more like a big HOWTO than a
distribution.

> > Although it lets the user 'escape' their original system quite quickly, my
> > interest lies more in creating a custom Linux installation using an
> > existing development environment.
> 
> Earlier versions of Linux From Scratch did that; they didn't have the 
> intermediate system, you just did a build from the main system to get a 
> result.  Unfortunately, it would do things like break strangely on certain 
> versions of debian due to environment variables or the way they'd patched 
> make, and they had to have various warnings and "if it does this, do this..."
> 
> The two stage thing of minimal build, chroot, build final system made it a lot 
> easier to reproduce regardless of the starting distro.

But it's a lot more difficult to use if the target system is a different
processor which is a lot less powerful than the development environment.

For example, say I wanted to install Linux on something like a VAX 11/780.

Clearly this isn't just going to be a case of put the CD in and press enter
a few times.

I'd much rather set up a cross development environment on this box, and create
a complete Linux-based system on a bootable SCSI disk that can then be plugged
in to the VAX than try to create a minimal environment in which to do the
construction of the system.

> > > As for the rest of your article: don't assume that trading a familiar set
> > > of problems for an unfamiliar set of problems automatically solves more
> > > than it causes.
> >
> > Well, I don't just assume that, but in this case, I think I've demonstrated
> > to myself that pre-compiled binaries are more difficult to support than
> > programs compiled from source.
> 
> And how many users are you trying to support?  (This varies depending on scale 
> of deployment.  Been there, done that...)

Not really sure what you mean, I don't have a specific, fixed userbase to
support.

> Being _able_ to rebuild the system with printf's in it is a Very Good Thing.  
> But presumably, that's what Red Hat's SRPMs are for...

It's not just physically whether the system was compiled from source on the
target machine or not, it's more a case of whether the end user, or in the
case of a company, somebody who is responsible for that department's
computing, made the decisions about what to install and how to install it,
based on their requirements, or whether they did a default installation,
whether that was appropriate for their needs or not.

> > However, surely that is just learning about more and more special cases.
> 
> All tech support is learning about more and more special cases.  You learn how 
> it _should_ work (which is often something you don't start out knowing, 
> because when it's working you don't HAVE to know).  You learn specific ways 
> it can break (by encountering them).  You learn to ask the right questions 
> when the user did something totally unrelated that broke it and doesn't think 
> to volunteer this information because it _IS_ seemingly totally unrelated 
> (like upgrading the kernel or libc, switching on ECN, moving the sucker 
> behind a masquerading router...)

I strongly, strongly, disagree here.

In my opinion this is a prime reason why users have so many problems with
many aspects of computing today.

Support should absolutely NOT be about learning specific problems and
specific solutions to them.

Just because it may seem to work 95% of the time is no excuse for justifying
what I consider to be a pathetic way of supporting any computing application.

It seems to work so well simply because this kind of support effectively
restricts how users can use their machines so much that it is possible for
"support staff" to learn most of the problems they are likely to encouter.

In turn, this devalues free, open source software, because it's removing one
of it's key advantages over proprietrary software - the ability to do highly
customised installtions with comparitive ease.

It's disgusting that users are effectively discouraged from using software
which isn't included in their chosen distribution, simply because they can't
get support.

> > It might even be possible to learn about such a large number of special
> > cases that a user can solve all the problems they encounter without any
> > generic knowledge of a system, but I think that is a bad thing to
> > encourage.
> 
> I.E. if the user becomes enough of an expert they don't need you to support 
> them anymore.

I find that quite insulting.

It seems to me that you are basically saying that users should stick to using
the arbitrary, limited building blocks that a typical distribution provides in
the form of pre-compiled binaries, just so that they are protected to a large
extent from messing up their systems to the degree that few people are skilled
enough to fix them.

Free software is about freedom - and yet you seem to be suggesting that users
place arbitrary restrictions upon themselves.

I believe that the existance of a highly bespoke support service, which is
available at fairly short notice most hours of the day, and charged at an
hourly rate, rather than a requiring a pre-existing support contract, helps
to remove such a restriction from users.

> 
> > If I get a phone call asking for help with a program I've never heard of
> > before, possibly on a distribution I've never heard of before, I belive
> > that I can offer much better generic advice based on an overall knowledge
> > of the system if everything is source-based, than if a distribution's
> > pre-compiled binaries have been installed.  To me, those pre-compiled
> > binaries are "non-standard".
> 
> Red Hat _is_ source based.  So is Debian.  The fact they build it on their 
> servers rather than the target machine doesn't mean they don't give you the 
> source to build it yourself if you want to.  However, figuring out how to 
> reproduce their build process can be a flaming pain.

See above - I said that it's not a case of which machine it's physically built
upon.

> The same is true with build-on-the-target-system distributions.  I put 
> together one based on a bash script, which is as simple as I could make it 
> and designed to have the bits cut and pasted out if you need to rebuild 
> stuff, but you've still got to remember to set the right environment 
> variables for source and target directories.

I believe that 'distributions' of any kind run the risk of encouraging the
bad support methods I described above.  This is not necessarily the fault of
the distributions - I am not particularly interested in apportioning blame,
rather I am pointing out the problem as I see it - support by learning
arbitrary specific cases is bad for the end user overall, in my opinion.

> > This is one reason why I do not advertise support services for proprietrary
> > systems.  I do not want to fix problems by learning about many, many,
> > special cases - I want to use my generic knowledge of computers to fix
> > problems, and I believe that it is a better way to use computers in
> > general.
> 
> I.E. you don't want to do tech support.

I don't want to do 'everyday' tech support.  My prices reflect that.

> Even with all the source, tech support is STILL knowing about many many 
> special cases.  Yes, you can work it out from first principles, and having 
> the source code in a readily buildable state where you can compile and drop 
> in functional replacements (this is NOT the same thing as just having the 
> source code) can help.
> 
> But if you're working out the answer from first principles every time you 
> provide somebody with tech support, you're going to be an amazingly 
> inefficient support person.

Less efficient than somebody who has seen that exact problem before, quite
possibly.

So, is your suggestion to users that they only cause problems that they know
their local technical support service is capable of fixing, and has seen
before?

If I can fix a user's machine, and practically nobody else can, or is
prepared to travel across the country in the middle of the night to do it
in time for business the next day, it's the user's choice whether they want
to pay for what you seem to describe as an "inefficient" support person, or
not.

Again, it's about offering a choice.  Nobody has an on-going support contract
with me.  Therefore, they don't have to worry about stepping outside what it
covers.

Ultimately there will always be a need for generically-knowledgable people to
solve some computing problems.

> > > Build from source has more potential, sure.  And when the standard
> > > laptop has a 10 ghz 64 bit processor and 4 gigabytes of ram, it'll make a
> > > lot of sense
> >
> > I don't think compiling from source is particularly inpractical on a
> > typical modern desktop machine today.
> 
> You've never built kde then.

Actually, my main desktop machine doesn't even have X installed.

However, I can build a 3.1-based kde-base and kde-libs in around an hour each
on a ~2 GHz machine with 512 Mb of RAM.

> > > But compiling stuff from source is HEAVILY dependent on sequencing;
> > > ./configure won't add ncurses support unless you installed ncurses first.
> >
> > That's true for a lot of libraries and basic components of a system, but I
> > don't think it's so much of a problem in general.
> 
> You've tried it, then?

I can't remember the last time I installed a binary from a distribution.

> > > How much stuff do you rebuild because you just added openssl, or switched
> > > to alsa from oss?
> >
> > Not too much to make it impractical, in my opinion.
> 
> And if the end-user decides to upgrade their system with stuff it didn't have 
> before?
> 
> This isn't restricted to source distributions, by the way.  I tried to update 
> a friend's SuSE system to the 2.6 kernel.  I expected to have to upgrade a 
> number of packages, but what I didn't expect was that SuSE doesn't seem to 
> have have ncurses installed (or at least not something make menuconfig can 
> find).  Python in Red Hat didn't have ncurses support last I checked, either.  
> (RH 9, I think...)
> 
> Suppose they don't select OpenSSL because they go "this is a desktop system, 
> not a server", and then they realise later "oh, I need https:// support in 
> Konqueror/Mozilla"...

Well, if a user installed everything, (or more or less everything), from
source, wouldn't they become more aware of the different options, which is
basically what I said right at the start?  It's the basic argument, yes, it
takes longer and is possibly slightly more tedious for a user to build from
source, but it's a good experience, and will teach them about the system,
saving time later on.

Plus, I offer support services to such users, to make it even more of a
practical option.

> > > How do you keep track of the dependencies so it CAN rebuild?
> >
> > In practice, I haven't found it to be particularly complicated.
> 
> How much practice have you found it in?

What?

> > > If your build system is a static command set like my shellscript, "build
> > > this list of packages in this order",
> >
> > It's not.  My 'build system' is basically me sitting at the console.
> 
> I've supported end-users before.  I've built systems for people who can't roll 
> their own.  Lots of the problems they have are non-obvious from the 
> perspective of geekdom.
> 
> > > how much of an improvement is this really
> > > over a binary distro?
> >
> > For the initial installation, maybe not much of an improvement, but for
> > on-going use of a system, I think compiling from source is better overall.
> 
> Name me a Linux distribution that is not "compiled from source".
> 
> You keep saying that installing from source is better, but it seems to be from 
> "gee, wouldn't it be nice if" land rather than due to actual experience.  You 
> _can_ build and install an SRPM into a Red Hat system.  It's too much of a 
> pain to be worth it to me, but it's been an option for years and years.

The initial discussion was about distributions and kernel development.

What I am trying to demonstrate is that users needn't be comitted to a
distribution at all.  They can use one as a base to do their own thing.

> Most desktop users I've seen who modify their own systems dirty them badly 
> enough that they copy their data off and reinstall every year or so anyway,

If the user only has one physical machine, I can understand the logic behind
that.
 
> so your long term support argument is a bit strange.

What long term support argument?

> having the server setup reproducible from a fresh 
> install is just plain good hygiene, anyway.  (Doesn't mean everybody does it, 
> but carrying around verbatim binary images through three or four hardware 
> upgrades is _not_ a recipe for maintainability.  And what do you do if the 
> sucker gets rootkitted, anyway?  Hope you got it all cleaned out?)

If a machine is root compromised, the system should be re-installed.

I'd usually suggest physically replacing the disks, re-installing, and
keeping the old disks for later analysis.

> I'm not saying compiling from source is BAD.  I'm just saying it's not as big 
> an improvement as you seem to think it will be.  It does allow a number of 
> new things to be done, but by itself it doesn't make a major difference.

Maybe because you are knowledgable enough to use the binaries provided by
a distribution, and supplement them with programs compiled from source where
necessary.  I think you probably do that without realising that many users
don't see compiling from source as a realistic option for them, and are
therefore stuck with what distributions provide.

Basically it boils down to arbitrary limits.  I am trying to remove them.
Free, open source software is available to everybody, but many people use
it in the same way as proprietrary software - 'off the shelf' binaries.  I
think that that represents a missed opportunity.

John.

Re: Distributions vs kernel development

[Jon Portnoy]

On Wed, 19 May 2004, Rob Landley wrote:

> 
> It's not really a distro.  It's an enormous HOWTO.  (Then again, so's Gentoo, 
> but gentoo does claim to be a distro, which is where I get disappointed...)
> 

No less a distribution than, say, Debian. You just type 'emerge' rather 
than 'apt-get' and get source tarballs rather than binary packages.

> 
> And then there's gentoo, which has a python script talk to a server out on the 
> net to figure out what to build.  If you're going to even TRIGGER that, you 
> need to be familiar with their packaging tool.  To take it apart and modify 
> the build would take a lot of eyeballing.
> 

Not quite; ebuilds are all on-disk. The only time you talk to a server is 
to update the on-disk ebuild tree (via rsync) or download a source 
tarball. Pretty much the same way BSD ports works. Taking apart and 
modifying the build is pretty trivial thanks to the ebuild(1) tool and the 
fact that ebuilds are in bash with Portage extensions.

> 
> Suppose they don't select OpenSSL because they go "this is a desktop system, 
> not a server", and then they realise later "oh, I need https:// support in 
> Konqueror/Mozilla"...
> 

Gentoo solves this problem with USE flags by providing a reasonable 
default set and letting users fine-tune the support they want prior to 
building the system.

> 
> You keep saying that installing from source is better, but it seems to be from 
> "gee, wouldn't it be nice if" land rather than due to actual experience.  You 
> _can_ build and install an SRPM into a Red Hat system.  It's too much of a 
> pain to be worth it to me, but it's been an option for years and years.
> 

The advantage, in my view, of compiling from source consistently is that 
you (a) eliminate any potential bugs from the build system being 
drastically different from the target system and (b) have the flexibility 
of being able to fine-tune dependencies and build time configuration. 
Where's the RPM package for Mozilla with encryption, without debugging, 
with gtk2, with ipv6, without ldap, without the calendar, with mail, 
without IRC, and without XFT? How about GCC with gcj, without f77, without 
nls, with objc?

It's certainly not for everybody, but to me that's the most important 
aspect of always compiling from source.

Re: Distributions vs kernel development

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 578 bytes --]

On Sun, 2004-05-09 at 11:13, Rene Rebe wrote:

> But the last time I took a look not even an installer or such. +
> Gentoo has no support for custom modifications not even thinking about
> a way to group such custom modifications / build configuration into a
> well defined way to form a distribution. + ROCK Linux has a real
> sandbox build environment, not this optimization via CFLAGS, and so on
> Gentoo wannabe.
> 

Weird .. last time I checked, Gentoo sandbox was originally derived from
ROCK Linux's sandbox wrapper ....


Cheers,

-- 
Martin Schlemmer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Distributions vs kernel development

[Rene Rebe]

Hi,

On: Mon, 24 May 2004 21:17:21 +0200,
    Martin Schlemmer <azarah@nosferatu.za.org> wrote:
> On Sun, 2004-05-09 at 11:13, Rene Rebe wrote:
> 
> > But the last time I took a look not even an installer or such. +
> > Gentoo has no support for custom modifications not even thinking about
> > a way to group such custom modifications / build configuration into a
> > well defined way to form a distribution. + ROCK Linux has a real
> > sandbox build environment, not this optimization via CFLAGS, and so on
> > Gentoo wannabe.
> > 
> 
> Weird .. last time I checked, Gentoo sandbox was originally derived from
> ROCK Linux's sandbox wrapper ....

Interesting ,) I did not know Gentoo copied ROCK Linux code, too. I
already put re-reviewing Gentoo on my this-year's TODO. So I'm finally
up-to-date with such stuff again ...

Sincerely yours,
  René Rebe
    - ROCK Linux stable release maintainer

--  
René Rebe - Europe/Germany/Berlin
  rene@rocklinux.org rene@rocklinux-consulting.de
http://www.rocklinux.org http://www.rocklinux-consulting.de

Re: Distributions vs kernel development

[John Bradford]

Hi,

Quote from Jon Portnoy <portnoy@tellink.net>:
> On Wed, 19 May 2004, Rob Landley wrote:
> 
> > 
> > It's not really a distro.  It's an enormous HOWTO.  (Then again, so's Gentoo, 
> > but gentoo does claim to be a distro, which is where I get disappointed...)
> > 
> 
> No less a distribution than, say, Debian. You just type 'emerge' rather 
> than 'apt-get' and get source tarballs rather than binary packages.

I think the "It's not really a distro" comment was in reference to Linux
>From Scratch.

[snip]

> > You keep saying that installing from source is better, but it seems to be from 
> > "gee, wouldn't it be nice if" land rather than due to actual experience.  You 
> > _can_ build and install an SRPM into a Red Hat system.  It's too much of a 
> > pain to be worth it to me, but it's been an option for years and years.
> > 
> 
> The advantage, in my view, of compiling from source consistently is that 
> you (a) eliminate any potential bugs from the build system being 
> drastically different from the target system and (b) have the flexibility 
> of being able to fine-tune dependencies and build time configuration. 

When I was talking about users compiling from source rather than using
distribution's pre-built binaries, I didn't really have in mind users
downloading a pre-written script that compiles a binary on their own machine.

In my opinion, there are two main ways to run a Linux-based system - either
follow a particular distribution, using their binaries, or source releases, or
work independently of any distribution, and do your own thing, obtaining the
source code for the software you want to use direct from it's ftp site, or
whatever method of distribution the authors use.

Of course, it's a long process to build a system completely from source.
In practice, many users might decide to use a distribution to install the
basic components of their system, such as GCC, glibc, and various system
libraries, and then build everything else manually.  This is what I mean
by not 'following' a distribution, but simply using it as a base for the
user to do their own thing.

Now, there is the issue of support.

In my opinion, in essence, there are two ways of offering support, (regardless
of whether it's free of charge or not, and whether it's provided by users,
via mailing lists, or by an organisation such as a distribution, or
independent company).

Firstly, it is possible to learn many of the common problems that can be
encountered with any particular distribution's pre-compiled binaries, or
binaries compiled on the user's machine from a fixed script.

Secondly, it is possible to become generically knowledgable about
computers, and to solve problems in a generic way, without specific
prior knowledge of a particular system.

In my opinion, the first option is the most widespread and the most encouraged
simply because it is much easier than the second option, and appears to work
most of the time.

I admit that if the user's problem is actually a common one, the first option
may be the fastest way to solve the immediate problem.

However, in my opinion, this whole approach only appears to work at all,
because it effectively limits many, (if not most), users to using their
computers in a fraction of the number of ways possible.

Or, to put it another way, users are effectively being forced to choose
less efficient configurations, just so that they can get support.

See my previous posts in this thread for more discussion of this.

John.

Re: Distributions vs kernel development

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1329 bytes --]

On Tue, 2004-05-25 at 10:22, Rene Rebe wrote:
> Hi,
> 
> On: Mon, 24 May 2004 21:17:21 +0200,
>     Martin Schlemmer <azarah@nosferatu.za.org> wrote:
> > On Sun, 2004-05-09 at 11:13, Rene Rebe wrote:
> > 
> > > But the last time I took a look not even an installer or such. +
> > > Gentoo has no support for custom modifications not even thinking about
> > > a way to group such custom modifications / build configuration into a
> > > well defined way to form a distribution. + ROCK Linux has a real
> > > sandbox build environment, not this optimization via CFLAGS, and so on
> > > Gentoo wannabe.
> > > 
> > 
> > Weird .. last time I checked, Gentoo sandbox was originally derived from
> > ROCK Linux's sandbox wrapper ....
> 
> Interesting ,) I did not know Gentoo copied ROCK Linux code, too.

Yeah, well, started out that way, as G. Bevin who wrote it at the
time, came from ROCK Linux (user I think).  I do not think it resembles
it much, if at all anymore though, and I know some additions have
been added that you guys might think about, like also hooking execve
(meaning subsequent bash's will still have LD_PRELOAD=sandboxlib
 in env, and if make/whatever sets LD_PRELOAD, the wrapper will add
 the sandboxlib back, etc), and one or two others I think.


Regards,

-- 
Martin Schlemmer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Distributions vs kernel development

[Chris Mason]

On Mon, 2004-05-10 at 11:39, James Morris wrote:
> On Fri, 7 May 2004, Andi Kleen wrote:
> 
> > The reiserfs attribute patch has been submitted many times,
> > but rejected for totally bogus reasons. You know who to complain to.
> 
> I'd really like to see the xattr + SELinux stuff go in, so that SELinux 
> users can have filesystem labeling under Reiserfs.  I'll volunteer to help 
> maintain this part of the code in mainline.

Thanks James.  Andrew wanted to wait until 2.6.7-pre before sending the
reiserfs acl bits to mainline; they have been sent up to Linus now.

-chris

Re: Distributions vs kernel development

[James Morris]

On Fri, 7 May 2004, Andi Kleen wrote:

> The reiserfs attribute patch has been submitted many times,
> but rejected for totally bogus reasons. You know who to complain to.

I'd really like to see the xattr + SELinux stuff go in, so that SELinux 
users can have filesystem labeling under Reiserfs.  I'll volunteer to help 
maintain this part of the code in mainline.


- James
-- 
James Morris
<jmorris@redhat.com>

Re: Distributions vs kernel development

[Pascal Schmidt]

On Fri, 07 May 2004 18:00:22 +0200, you wrote in linux.kernel:
> After having being burned twice: first by Mandrake and supermount, and second
> by SuSe and reiserfs attributes; are any of the distributions committed to
> making sure that their distribution will run the standard kernel? (ie. 2.6.X from
> kernel.org).

Slackware, both -current and 9.1. Don't expect fancy initscripts
handling every possible thing you could boot off for you, though.

-- 
Ciao,
Pascal

Re: Distributions vs kernel development

[Andi Kleen]

Stephen Hemminger <shemminger@osdl.org> writes:

> After having being burned twice: first by Mandrake and supermount, and second
> by SuSe and reiserfs attributes; are any of the distributions committed to
> making sure that their distribution will run the standard kernel? (ie. 2.6.X from
> kernel.org). When running a non-vendor kernel, I need to reasonably expect that the system
> will boot and all the filesystems and standard devices are available.  I don't
> expect every startup script to run clean, or every device that has a driver
> only in the vendor kernel to work. 

The reiserfs attribute patch has been submitted many times,
but rejected for totally bogus reasons. You know who to complain to.

-Andi