How to tell whether a struct file is held by a process?

How to tell whether a struct file is held by a process?

[Alan Stern]

What's the best way to tell whether the current process has a
particular struct file among its open files?  Is there any better way
to find out than blindly calling fget() for each possible fd?

Is this a totally insane thing to do?

Alan Stern

Re: How to tell whether a struct file is held by a process?

[Al Viro]

On Tue, May 19, 2009 at 12:57:21PM -0400, Alan Stern wrote:
> What's the best way to tell whether the current process has a
> particular struct file among its open files?  Is there any better way
> to find out than blindly calling fget() for each possible fd?
> 
> Is this a totally insane thing to do?

It is insane.  You might lock fdtable and scan it, but as soon as you
drop the spinlock your return value is worthless.

What are you trying to do?  If the process is cooperating, you don't really
need that in the kernel, if it's not, the check is not usable...

Re: How to tell whether a struct file is held by a process?

[Alan Stern]

On Thu, 21 May 2009, Al Viro wrote:

> On Tue, May 19, 2009 at 12:57:21PM -0400, Alan Stern wrote:
> > What's the best way to tell whether the current process has a
> > particular struct file among its open files?  Is there any better way
> > to find out than blindly calling fget() for each possible fd?
> > 
> > Is this a totally insane thing to do?
> 
> It is insane.  You might lock fdtable and scan it, but as soon as you
> drop the spinlock your return value is worthless.

In this case, I believe that changes after the check has been made 
won't hurt -- everything relevant to my work will be serialized by a 
separate lock.

> What are you trying to do?  If the process is cooperating, you don't really
> need that in the kernel, if it's not, the check is not usable...

I'm trying to work out a good way to reserve access rights to a device
-- even if that device doesn't exist yet.

Here's the story.  People have requested that the kernel add a
mechanism whereby a user program can get more-or-less exclusive access
rights to a USB device.  In fact, they'd like to reserve these rights
for any device plugged into a particular USB port.  So even if no
device is plugged into that port at the moment, the program should get
exclusive access as soon as a new device is detected there.

In order to prevent programs from dying without releasing their
exclusive rights, it seems natural to implement these rights as open
files.  Thus, opening file A(P) will give a program exclusive access
rights to any device plugged into USB port P.  Closing A(P) releases
the rights.  (The A(P) files would be implemented as single-open files, 
probably in sysfs.)

The problem is this.  Let D be the device plugged into port P.  When
some program opens D's device file, it's necessary to check whether
that same program has an open file reference for A(P), i.e., has opened
A(P), or has inherited a descriptor for A(P) from its parent, or has
been passed such a descriptor over a Unix socket, etc.  If not, and if 
A(P) is open (owned by someone else), then access to D is denied.

If you can think up a better way to implement these exclusive access 
rights, I'd be glad to hear it.

Alan Stern

Re: How to tell whether a struct file is held by a process?

[Kay Sievers]

On Thu, May 21, 2009 at 16:06, Alan Stern <stern@rowland.harvard.edu> wrote:
> On Thu, 21 May 2009, Al Viro wrote:
>> On Tue, May 19, 2009 at 12:57:21PM -0400, Alan Stern wrote:
>> > What's the best way to tell whether the current process has a
>> > particular struct file among its open files?  Is there any better way
>> > to find out than blindly calling fget() for each possible fd?
>> >
>> > Is this a totally insane thing to do?
>>
>> It is insane.  You might lock fdtable and scan it, but as soon as you
>> drop the spinlock your return value is worthless.
>
> In this case, I believe that changes after the check has been made
> won't hurt -- everything relevant to my work will be serialized by a
> separate lock.
>
>> What are you trying to do?  If the process is cooperating, you don't really
>> need that in the kernel, if it's not, the check is not usable...
>
> I'm trying to work out a good way to reserve access rights to a device
> -- even if that device doesn't exist yet.
>
> Here's the story.  People have requested that the kernel add a
> mechanism whereby a user program can get more-or-less exclusive access
> rights to a USB device.  In fact, they'd like to reserve these rights
> for any device plugged into a particular USB port.  So even if no
> device is plugged into that port at the moment, the program should get
> exclusive access as soon as a new device is detected there.
>
> In order to prevent programs from dying without releasing their
> exclusive rights, it seems natural to implement these rights as open
> files.  Thus, opening file A(P) will give a program exclusive access
> rights to any device plugged into USB port P.  Closing A(P) releases
> the rights.  (The A(P) files would be implemented as single-open files,
> probably in sysfs.)
>
> The problem is this.  Let D be the device plugged into port P.  When
> some program opens D's device file, it's necessary to check whether
> that same program has an open file reference for A(P), i.e., has opened
> A(P), or has inherited a descriptor for A(P) from its parent, or has
> been passed such a descriptor over a Unix socket, etc.  If not, and if
> A(P) is open (owned by someone else), then access to D is denied.
>
> If you can think up a better way to implement these exclusive access
> rights, I'd be glad to hear it.

Not sure, if we already discussed that with a conclusion: this
"prevent access to a future device" interface needs to work at the pid
level, or would a uid/gid check be sufficient?

Root can do all that stuff anyway, even with the locking in place.
Uid/gid file permissions need to be applied to the "lock file", so
specified non-root users can use that interface.

Maybe it would be good enough, to check that the one that opened the
"lock file" has the same uid/gid as the on that tries to open the
device when it has shown up?

Kay

Re: How to tell whether a struct file is held by a process?

[Alan Stern]

On Thu, 21 May 2009, Kay Sievers wrote:

> Not sure, if we already discussed that with a conclusion: this
> "prevent access to a future device" interface needs to work at the pid
> level, or would a uid/gid check be sufficient?
> 
> Root can do all that stuff anyway, even with the locking in place.
> Uid/gid file permissions need to be applied to the "lock file", so
> specified non-root users can use that interface.
> 
> Maybe it would be good enough, to check that the one that opened the
> "lock file" has the same uid/gid as the on that tries to open the
> device when it has shown up?

I don't know; it depends on what people want.  Pantelis, would this be 
good enough for you?  That is, restrict the device either to programs 
having the same uid/gid or having the same pid as the process that 
opened the lock file?

(Note that the pid check alone isn't fully secure, since pid values get 
reused after a process ends.)

Alan Stern

Re: How to tell whether a struct file is held by a process?

[Alan Cox]

> Not sure, if we already discussed that with a conclusion: this
> "prevent access to a future device" interface needs to work at the pid
> level, or would a uid/gid check be sufficient?

As I said before neither pid or uid make any sense at all for a kernel
level check beyond what already exists at user space level (ie chown),
and what the usual user space gunk provides for device creation policy
(udev)

> Root can do all that stuff anyway, even with the locking in place.
> Uid/gid file permissions need to be applied to the "lock file", so
> specified non-root users can use that interface.

It's actually sometimes important that root can ignore the lock
files - eg to send a reset to a channel of a jammed device that is owned
by a user and wedged.
 
> Maybe it would be good enough, to check that the one that opened the
> "lock file" has the same uid/gid as the on that tries to open the
> device when it has shown up?

My feeling too - there simply isn't any need for kernel infrastructure
here. The only case the kernel must be involved is when you need
exclusivity including telling the kernel to leave things alone, and even
that usually only needs to be by specific handle and we normally borrow
O_EXCL for it as it has no "traditional" meaning on device nodes.

We do tty devices with lock files no reason it won't work for USB stuff
too.

Alan

Re: How to tell whether a struct file is held by a process?

[Alan Cox]

> Here's the story.  People have requested that the kernel add a
> mechanism whereby a user program can get more-or-less exclusive access
> rights to a USB device.  In fact, they'd like to reserve these rights

People ask for lots of things, often things they should be doing user
space, violate the laws of causality or are just dumb.

> In order to prevent programs from dying without releasing their
> exclusive rights, it seems natural to implement these rights as open
> files.  Thus, opening file A(P) will give a program exclusive access

Standard unix lock files have this property and its worked for many
things for many years.

> The problem is this.  Let D be the device plugged into port P.  When
> some program opens D's device file, it's necessary to check whether
> that same program has an open file reference for A(P), i.e., has opened
> A(P), or has inherited a descriptor for A(P) from its parent, or has
> been passed such a descriptor over a Unix socket, etc.  If not, and if 
> A(P) is open (owned by someone else), then access to D is denied.

Define "program" - is that one thread, many threads, a group of processes
involving child processes (often essential for user space device drivers
to get the security boundaries right) ? pid doesn't work for thread
groups or security divisisons, uid doesn't work for security.

There are two kernel models used that I can think of
	- lock file (pure user space policy)
	- O_EXCL - stop any other opens with -EBUSY

the latter plus file handle passing is quite sufficient to implement
pretty much any policy cleanly and to ensure that there are no
"inadvertent" accessors. It does mean the fd has to be passed about but
that is easy and the natural flow of apps doing that kind of work is to
inherit such handles.

> If you can think up a better way to implement these exclusive access 
> rights, I'd be glad to hear it.

User space - precisely because defining the access policy and what its
really meant by "program" in this case is so device and situation
dependant.

Re: How to tell whether a struct file is held by a process?

[Kyle Moffett]

On Thu, May 21, 2009 at 6:14 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> Not sure, if we already discussed that with a conclusion: this
>> "prevent access to a future device" interface needs to work at the pid
>> level, or would a uid/gid check be sufficient?
>
> As I said before neither pid or uid make any sense at all for a kernel
> level check beyond what already exists at user space level (ie chown),
> and what the usual user space gunk provides for device creation policy
> (udev)
>
>> Root can do all that stuff anyway, even with the locking in place.
>> Uid/gid file permissions need to be applied to the "lock file", so
>> specified non-root users can use that interface.
>
> It's actually sometimes important that root can ignore the lock
> files - eg to send a reset to a channel of a jammed device that is owned
> by a user and wedged.
>
>> Maybe it would be good enough, to check that the one that opened the
>> "lock file" has the same uid/gid as the on that tries to open the
>> device when it has shown up?
>
> My feeling too - there simply isn't any need for kernel infrastructure
> here. The only case the kernel must be involved is when you need
> exclusivity including telling the kernel to leave things alone, and even
> that usually only needs to be by specific handle and we normally borrow
> O_EXCL for it as it has no "traditional" meaning on device nodes.
>
> We do tty devices with lock files no reason it won't work for USB stuff
> too.

Hrm, I think this is missing the point.  Here's a practical example for you:

My Debian laptop has a Windows VM (using KVM, specifically) on it for
the occasional times when work forces me to use such software.  One of
the things I need the Windows VM for is a USB JTAG debug box that (for
stupid Windows driver reasons) shows up as an HID device.  The debug
box needs firmware loaded on it as soon as it is attached or it tends
to crash or behave unreliably; the firmware loading software is also
Windows-only.  There are also other random USB devices like encrypted
flash drives with windows-only software, software license tokens, etc.

My ideal solution is to be able to prevent Linux from automatically
binding to any devices attached off specific USB ports.  There's
simply no point in letting the usb-storage or usb-hid modules try to
scan those devices because I'm just going to immediately "unbind" them
and drive them with libusb *anyways*.

Basically I want one of my USB ports (which I plug into an external
hub) to always go to my Windows KVM, and another USB port (also with a
hub) to always go to my Fedora KVM.  There are a couple requirements
for this:

(1)  I need to be able to exclude any access *BEFORE* a device is connected.
(2)  This exclusion needs to apply to kernel drivers

One possible alternative (although I don't know if this would work for
the OP), is to make there be a global "prevent kernel drivers from
autobinding to  devices on his port" flag, and manage the rest with
lockfiles in userspace.

Cheers,
Kyle Moffett

Re: How to tell whether a struct file is held by a process?

[Oliver Neukum]

Am Freitag, 22. Mai 2009 00:14:52 schrieb Alan Cox:
> My feeling too - there simply isn't any need for kernel infrastructure
> here. The only case the kernel must be involved is when you need

That seems a reasonable solution to me.

	Regards
		Oliver

Re: How to tell whether a struct file is held by a process?

[Alan Cox]

> the OP), is to make there be a global "prevent kernel drivers from
> autobinding to  devices on his port" flag, and manage the rest with
> lockfiles in userspace.

Your "practical example" seems to bear no relation to the other stuff,
its a separate topic altogether.

Yes a "don't probe this port" would make sense. The fact kernel space
probes the USB devices by default like the fact we probe partitions by
default and scan scsi busses by default are all really mistakes inherited
from traditional OS designs which also mess up virtualisation the same
way.

So add a skip_ports= facility to the usb core code (or a general
auto_enumerate=0 and a udev interface to trigger scanning so your udev
rules can enumerate just the ports you want). That is probably cleaner
because it makes the identification of ports rather cleaner and clearer.

Alan

Re: How to tell whether a struct file is held by a process?

[Oliver Neukum]

Am Thursday 21 May 2009 16:06:00 schrieben Sie:
> The problem is this.  Let D be the device plugged into port P.  When
> some program opens D's device file, it's necessary to check whether
> that same program has an open file reference for A(P), i.e., has opened

No. It is merely nice to do that, not necessary. You can take the position
that if the port is reserved the kernel won't touch the device but it is
user space's responsibility to not touch a device user space wants
to reserve.

Or alternatively, don't create the device file until user space tells
you to do so.

	Regards
		Oliver

Re: How to tell whether a struct file is held by a process?

[Pantelis Koukousoulas]

The only problem I find with leaving mutual exclusion 100% to userspace and
burdening the kernel only with the decision of whether a port should be
handled by kernel or userspace is this:

Suppose a device needs a reset as part of its init sequence (a whole lot
of them do, this is not purely hypothetical). Then a different process may
get to operate the device before and after the reset and hilarity may result
from that.

There is also the issue of a Program (as in Vmware, Qemu etc, not as in pid)
thinking it has claimed a port and then finding the device just connected
there is actually unavailable to operate, but this doesn't sound that bad.
(After all, the device itself might have been broken, having claimed a port
in no way guarantees success in operating devices connected there).

So, if there is a clean / acceptable way to handle the reset issue in userspace
I 'm happy to dispose with kernel-level checks for 'allowed processes'
altogether.

Does that sound reasonable?

Thanks a lot for the discussion,
Pantelis

Re: How to tell whether a struct file is held by a process?

[Oliver Neukum]

Am Freitag, 22. Mai 2009 15:30:20 schrieb Pantelis Koukousoulas:
> The only problem I find with leaving mutual exclusion 100% to userspace and
> burdening the kernel only with the decision of whether a port should be
> handled by kernel or userspace is this:
>
> Suppose a device needs a reset as part of its init sequence (a whole lot
> of them do, this is not purely hypothetical). Then a different process may
> get to operate the device before and after the reset and hilarity may
> result from that.

If you leave the locking against user space to user space this can happen
any time, not just due to a reset. What is so special about reset? You just
need a user space locking scheme operating on port numbers, not device
addresses.

	Regards
		Oliver

Re: How to tell whether a struct file is held by a process?

[Alan Cox]

> Suppose a device needs a reset as part of its init sequence (a whole lot
> of them do, this is not purely hypothetical). Then a different process may
> get to operate the device before and after the reset and hilarity may result
> from that.

Thats surely up to you to get your lock file usage right. The kernel
isn't there to play mother to crap programming.

> So, if there is a clean / acceptable way to handle the reset issue in userspace

Firstly can you explain *why* you think there is a problem ?

Re: How to tell whether a struct file is held by a process?

[Pantelis Koukousoulas]

On Fri, May 22, 2009 at 4:43 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> Suppose a device needs a reset as part of its init sequence (a whole lot
>> of them do, this is not purely hypothetical). Then a different process may
>> get to operate the device before and after the reset and hilarity may result
>> from that.
>
> Thats surely up to you to get your lock file usage right. The kernel
> isn't there to play mother to crap programming.

Who is "me" in this case? Assuming that those writing the userspace programs
are cooperative though, I agree. We can declare that whatever program does not
play by the (userspace locking) rules is crap and either fix it (if open source)
or refuse to install it / complain (if closed source).

>
>> So, if there is a clean / acceptable way to handle the reset issue in userspace
>
> Firstly can you explain *why* you think there is a problem ?
>

I admit this could be a bias from the way I imagined the whole thing
to work before
this discussion. I think I can see how a userspace locking scheme based on port
numbers could avoid also the reset problem.

Thanks,
Pantelis

Re: How to tell whether a struct file is held by a process?

[Alan Cox]

> Who is "me" in this case? Assuming that those writing the userspace programs
> are cooperative though, I agree. We can declare that whatever program does not

The software authors.

> play by the (userspace locking) rules is crap and either fix it (if open source)
> or refuse to install it / complain (if closed source).

Which for the general case I think is reasonable. If the bad program is
determined to be bad then it can equally just ptrace the process with the
file open and drive it that way.

> >
> >> So, if there is a clean / acceptable way to handle the reset issue in userspace
> >
> > Firstly can you explain *why* you think there is a problem ?
> >
> 
> I admit this could be a bias from the way I imagined the whole thing
> to work before
> this discussion. I think I can see how a userspace locking scheme based on port
> numbers could avoid also the reset problem

That was a serious request to understand what you think the problem is
and what problem is worrying you. I'm deeply sceptical of the need for
any kernel locking on this one but I'd like to understand better why you
think there are some cases it would be needed.

Re: How to tell whether a struct file is held by a process?

[Pantelis Koukousoulas]

>> > Firstly can you explain *why* you think there is a problem ?
>> >
>>
>> I admit this could be a bias from the way I imagined the whole thing
>> to work before
>> this discussion. I think I can see how a userspace locking scheme based on port
>> numbers could avoid also the reset problem
>
> That was a serious request to understand what you think the problem is
> and what problem is worrying you. I'm deeply sceptical of the need for
> any kernel locking on this one but I'd like to understand better why you
> think there are some cases it would be needed.
>

I don't anymore think kernel locking is really needed for this part. I 'm sorry
if my answer did not sound as serious as your request.

So, as long as the kernel has a facility to allow assigning ports to userspace,
the rest can be arranged between the userspace processes in a cooperative
manner (and in fact this seems to be the only sufficiently clean way from what
I can tell).

Thanks,
Pantelis

Re: How to tell whether a struct file is held by a process?

[Alan Stern]

On Fri, 22 May 2009, Alan Cox wrote:

> > the OP), is to make there be a global "prevent kernel drivers from
> > autobinding to  devices on his port" flag, and manage the rest with
> > lockfiles in userspace.
> 
> Your "practical example" seems to bear no relation to the other stuff,
> its a separate topic altogether.
> 
> Yes a "don't probe this port" would make sense. The fact kernel space
> probes the USB devices by default like the fact we probe partitions by
> default and scan scsi busses by default are all really mistakes inherited
> from traditional OS designs which also mess up virtualisation the same
> way.

I don't understand this comment.  By "probe" I assume you mean
"initialize and enumerate", as opposed to calling various drivers'
probe() routines -- let me know if this isn't what you meant.  If the
kernel didn't initialize and enumerate devices by default, then when
would they get initialized and enumerated?

If your answer is "When requested by a user process", then I'll go on
to ask: What about devices containing the root filesystem, the system 
console, and so on?

And just how does initialization and enumeration mess up 
virtualization?

> So add a skip_ports= facility to the usb core code (or a general
> auto_enumerate=0 and a udev interface to trigger scanning so your udev
> rules can enumerate just the ports you want). That is probably cleaner
> because it makes the identification of ports rather cleaner and clearer.

Anyway, enumeration isn't the problem.  The real problem has two parts:

	Automatic probing and binding of kernel drivers, including
	selection and installation of a configuration (this really
	_does_ mess up virtualization).

	The fact that a window exists immediately after the 
	registration of a newly-detected device before a user
	process can lock the device file.  During this window,
	other processes could open the file.

The second part can be solved (among cooperating processes) by use of
port-lock files, with no kernel involvement.  The first part does
require a kernel interface of some sort, but it wouldn't have to be
complicated.  The mere fact that a port-lock file was open could be
enough to prevent automatic configuration, probing, and binding.

Does this seem like reasonable approach?

Alan Stern

Re: How to tell whether a struct file is held by a process?

[Alan Cox]

> I don't understand this comment.  By "probe" I assume you mean
> "initialize and enumerate", as opposed to calling various drivers'
> probe() routines -- let me know if this isn't what you meant.  If the
> kernel didn't initialize and enumerate devices by default, then when
> would they get initialized and enumerated?

When user space gets an event saying the bus itself exists and asks for
it to be scanned. The most obvious case where its currently broken is
partition tables on disks where you don't always want to read partition
tables automatically.

> 
> If your answer is "When requested by a user process", then I'll go on
> to ask: What about devices containing the root filesystem, the system 
> console, and so on?
> 
> And just how does initialization and enumeration mess up 
> virtualization?

It means that the bus scanning occurs out of control of the virtualised
environment which might want to control what occurs

> Anyway, enumeration isn't the problem.  The real problem has two parts:
> 
> 	Automatic probing and binding of kernel drivers, including
> 	selection and installation of a configuration (this really
> 	_does_ mess up virtualization).

If enumeration isn't automatic (or even more so if binding isn't always
automatic) then the problem doesn't occur.

> The second part can be solved (among cooperating processes) by use of
> port-lock files, with no kernel involvement.  The first part does
> require a kernel interface of some sort, but it wouldn't have to be
> complicated.  The mere fact that a port-lock file was open could be
> enough to prevent automatic configuration, probing, and binding.

Given the layout of devices can change fairly arbitarily would the
ability to claim specific device identifiers via libusb do the job any
better.

ie would it be better to expose an interface via libusb that was
essentially

	reserve_for_userspace(vendorid, devid);

Re: How to tell whether a struct file is held by a process?

[Kay Sievers]

On Fri, May 22, 2009 at 17:12, Alan Stern <stern@rowland.harvard.edu> wrote:
> Anyway, enumeration isn't the problem.  The real problem has two parts:
>
>        Automatic probing and binding of kernel drivers, including
>        selection and installation of a configuration (this really
>        _does_ mess up virtualization).
>
>        The fact that a window exists immediately after the
>        registration of a newly-detected device before a user
>        process can lock the device file.  During this window,
>        other processes could open the file.
>
> The second part can be solved (among cooperating processes) by use of
> port-lock files, with no kernel involvement.  The first part does
> require a kernel interface of some sort, but it wouldn't have to be
> complicated.  The mere fact that a port-lock file was open could be
> enough to prevent automatic configuration, probing, and binding.
>
> Does this seem like reasonable approach?

Would releasing the "lock" trigger a kernel-driver-binding call?

The lock will always lock all devices of a specific hub?

Thanks,
Kay

Re: How to tell whether a struct file is held by a process?

[Alan Stern]

On Fri, 22 May 2009, Kay Sievers wrote:

> On Fri, May 22, 2009 at 17:12, Alan Stern <stern@rowland.harvard.edu> wrote:
> > Anyway, enumeration isn't the problem.  The real problem has two parts:
> >
> >        Automatic probing and binding of kernel drivers, including
> >        selection and installation of a configuration (this really
> >        _does_ mess up virtualization).
> >
> >        The fact that a window exists immediately after the
> >        registration of a newly-detected device before a user
> >        process can lock the device file.  During this window,
> >        other processes could open the file.
> >
> > The second part can be solved (among cooperating processes) by use of
> > port-lock files, with no kernel involvement.  The first part does
> > require a kernel interface of some sort, but it wouldn't have to be
> > complicated.  The mere fact that a port-lock file was open could be
> > enough to prevent automatic configuration, probing, and binding.
> >
> > Does this seem like reasonable approach?
> 
> Would releasing the "lock" trigger a kernel-driver-binding call?

No.  If the lock owner wants to bind kernel drivers, it can use the
existing API in libusb after releasing the lock.  This might cause
problems if the owning process terminates abnormally, but I think we
can live with that.

> The lock will always lock all devices of a specific hub?

The idea is that there will be one lock file per port.  So for example, 
a hub device with four ports might contain inside its sysfs device 
directory: ports/1, ..., ports/4.

Alan Stern

Re: How to tell whether a struct file is held by a process?

[Alan Stern]

On Fri, 22 May 2009, Alan Cox wrote:

> > And just how does initialization and enumeration mess up 
> > virtualization?
> 
> It means that the bus scanning occurs out of control of the virtualised
> environment which might want to control what occurs

With USB this isn't much of an issue.  There are almost no
device-specific parameters affecting enumeration and none affecting
initialization -- understandably, since before initialization the
kernel doesn't know what kind of device it's dealing with.

> > Anyway, enumeration isn't the problem.  The real problem has two parts:
> > 
> > 	Automatic probing and binding of kernel drivers, including
> > 	selection and installation of a configuration (this really
> > 	_does_ mess up virtualization).
> 
> If enumeration isn't automatic (or even more so if binding isn't always
> automatic) then the problem doesn't occur.

I disagree.  For one thing, probing of kernel drivers doesn't occur
during enumeration.  So we could have automatic enumeration without
automatic probing, and there would be no problem.  For another, even if
enumeration were the direct cause of some problem, it would still
cause that same problem whenever it occurred, whether automatically or
not.

> > The second part can be solved (among cooperating processes) by use of
> > port-lock files, with no kernel involvement.  The first part does
> > require a kernel interface of some sort, but it wouldn't have to be
> > complicated.  The mere fact that a port-lock file was open could be
> > enough to prevent automatic configuration, probing, and binding.
> 
> Given the layout of devices can change fairly arbitarily would the
> ability to claim specific device identifiers via libusb do the job any
> better.
> 
> ie would it be better to expose an interface via libusb that was
> essentially
> 
> 	reserve_for_userspace(vendorid, devid);

That's a good question.  I'll defer to the interested parties.  
Kyle and Pantelis, what do you think?

Alan Stern

Re: How to tell whether a struct file is held by a process?

[Pantelis Koukousoulas]

>> > The second part can be solved (among cooperating processes) by use of
>> > port-lock files, with no kernel involvement.  The first part does
>> > require a kernel interface of some sort, but it wouldn't have to be
>> > complicated.  The mere fact that a port-lock file was open could be
>> > enough to prevent automatic configuration, probing, and binding.
>>
>> Given the layout of devices can change fairly arbitarily would the
>> ability to claim specific device identifiers via libusb do the job any
>> better.
>>
>> ie would it be better to expose an interface via libusb that was
>> essentially
>>
>>       reserve_for_userspace(vendorid, devid);
>
> That's a good question.  I'll defer to the interested parties.
> Kyle and Pantelis, what do you think?

When a device needs a reset, it typically comes back with a different VID:PID.
So e.g., qemu would have to know both of those, while in the case
where we bind to ports it doesn't have to know either.
(This is good imho since the device may only have a windows driver and we
save the user from some research/experimentation).

I do understand that VID:PID solves the problem with port numbers changing
though, but I think in the end using ports is still easier for the
user with some
effort from the virtualization program implementor.

Note though, I assume that there is / will be a way to reliably
predict the port
number (at least roughly) given a physical port location. E.g., if I plugged
a device to my front left usb connector and it got port number X, it should
be possible to predict what the port number will be if I reconnect the same
device to the same connector. (This doesn't have to hold across reboots).

Pantelis

Re: How to tell whether a struct file is held by a process?

[Alan Stern]

On Fri, 22 May 2009, Pantelis Koukousoulas wrote:

> Note though, I assume that there is / will be a way to reliably
> predict the port
> number (at least roughly) given a physical port location. E.g., if I plugged
> a device to my front left usb connector and it got port number X, it should
> be possible to predict what the port number will be if I reconnect the same
> device to the same connector. (This doesn't have to hold across reboots).

If you don't reboot between connects, the port numbers and hub 
identifiers will not change.

Alan Stern

Re: How to tell whether a struct file is held by a process?

[Kay Sievers]

On Fri, May 22, 2009 at 17:54, Alan Stern <stern@rowland.harvard.edu> wrote:

>> Would releasing the "lock" trigger a kernel-driver-binding call?
>
> No.  If the lock owner wants to bind kernel drivers, it can use the
> existing API in libusb after releasing the lock.  This might cause
> problems if the owning process terminates abnormally, but I think we
> can live with that.
>
>> The lock will always lock all devices of a specific hub?
>
> The idea is that there will be one lock file per port.  So for example,
> a hub device with four ports might contain inside its sysfs device
> directory: ports/1, ..., ports/4.

Sounds both good to me.

Kay

Re: How to tell whether a struct file is held by a process?

[Pantelis Koukousoulas]

>> The idea is that there will be one lock file per port.  So for example,
>> a hub device with four ports might contain inside its sysfs device
>> directory: ports/1, ..., ports/4.
>
> Sounds both good to me.

Yes, I 'm happy with that solution too.

Thanks,
Pantelis

Re: How to tell whether a struct file is held by a process?

[Alan Stern]

On Fri, 22 May 2009, Kay Sievers wrote:

> On Fri, May 22, 2009 at 17:54, Alan Stern <stern@rowland.harvard.edu> wrote:
> 
> >> Would releasing the "lock" trigger a kernel-driver-binding call?
> >
> > No.  If the lock owner wants to bind kernel drivers, it can use the
> > existing API in libusb after releasing the lock.  This might cause
> > problems if the owning process terminates abnormally, but I think we
> > can live with that.
> >
> >> The lock will always lock all devices of a specific hub?
> >
> > The idea is that there will be one lock file per port.  So for example,
> > a hub device with four ports might contain inside its sysfs device
> > directory: ports/1, ..., ports/4.
> 
> Sounds both good to me.

Come to think of it, putting the lock files in sysfs isn't such a good 
idea.  The core will need to know whether the files are open, so we'll 
have to have our own file_operations structure for them.

Which means the best place to put the lock files is probably somewhere 
in /dev.  Can this be made to work by generating appropriate uevents, 
with the default udev rules?

Alan Stern

Re: How to tell whether a struct file is held by a process?

[Kay Sievers]

On Fri, May 22, 2009 at 21:31, Alan Stern <stern@rowland.harvard.edu> wrote:
> Come to think of it, putting the lock files in sysfs isn't such a good
> idea.  The core will need to know whether the files are open, so we'll
> have to have our own file_operations structure for them.
>
> Which means the best place to put the lock files is probably somewhere
> in /dev.  Can this be made to work by generating appropriate uevents,
> with the default udev rules?

Nodes in /dev would  need a corresponding device in /sys and belong to
a subsystem to send an event. That sounds like a lot of stuff for a
simple interface like this. Can't we just add ioctls to the hub device
nodes to implement the locking?

Thanks,
Kay

Re: How to tell whether a struct file is held by a process?

[Alan Stern]

On Fri, 22 May 2009, Kay Sievers wrote:

> On Fri, May 22, 2009 at 21:31, Alan Stern <stern@rowland.harvard.edu> wrote:
> > Come to think of it, putting the lock files in sysfs isn't such a good
> > idea.  The core will need to know whether the files are open, so we'll
> > have to have our own file_operations structure for them.
> >
> > Which means the best place to put the lock files is probably somewhere
> > in /dev.  Can this be made to work by generating appropriate uevents,
> > with the default udev rules?
> 
> Nodes in /dev would  need a corresponding device in /sys and belong to
> a subsystem to send an event. That sounds like a lot of stuff for a
> simple interface like this. Can't we just add ioctls to the hub device
> nodes to implement the locking?

Yes, that would work.  Closing the hub device file would release all 
the locked ports.

It wouldn't leave any lock files for user programs.  They would have to 
arrange the locking among themselves somehow.  Would that be okay?

Alan Stern

Re: How to tell whether a struct file is held by a process?

[Kay Sievers]

On Sat, May 23, 2009 at 00:07, Alan Stern <stern@rowland.harvard.edu> wrote:

>> Can't we just add ioctls to the hub device
>> nodes to implement the locking?
>
> Yes, that would work.  Closing the hub device file would release all
> the locked ports.
>
> It wouldn't leave any lock files for user programs.  They would have to
> arrange the locking among themselves somehow.  Would that be okay?

I would expect that userspace drivers competing about raw USB devices
wouldn't be that common. And I guess, most of these problems can
probably be solved by simple file permissions/ACLs applied to the USB
device nodes, in the same way the access to the hub device node would
be granted to lock-out the kernel drivers.

Kay