LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* Racy NLS behaviour in FAT (and possible other fs)
@ 2007-02-19 15:53 Pierre Ossman
  2007-02-19 16:55 ` OGAWA Hirofumi
  0 siblings, 1 reply; 5+ messages in thread
From: Pierre Ossman @ 2007-02-19 15:53 UTC (permalink / raw)
  To: hirofumi, LKML

Hi,

I'm experiencing a rather odd behaviour with the character set conversion. If I
mount a vfat fs with utf8 and then create a file with invalid utf-8 sequences,
the file will briefly exist with these invalid sequences, then quickly convert
to a stripped version.

I haven't found an easy way to catch the race, but if I have nautilus open it
tends to catch it now and then (I get a file name with "<?>" replacing each bad
byte).

The race also seems to corrupt the in-memory state of the fs now and then. I
managed to create a file where "ls" shows "?" for most fields. Data seemed to
have made it to disk ok though (fsck didn't complain and a remount showed
everything as it should be).

Third, there seems to be a problem with not all syscalls being subjected to the
NLS transformation. Example:

$ echo foo > baråäö.txt
$ ls
foo.txt
$ echo foo > baråäö.txt
bash: baråäö.txt: File exists

Rgds
-- 
     -- Pierre Ossman

  Linux kernel, MMC maintainer        http://www.kernel.org
  PulseAudio, core developer          http://pulseaudio.org
  rdesktop, core developer          http://www.rdesktop.org

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Racy NLS behaviour in FAT (and possible other fs)
  2007-02-19 15:53 Racy NLS behaviour in FAT (and possible other fs) Pierre Ossman
@ 2007-02-19 16:55 ` OGAWA Hirofumi
  2007-02-19 17:03   ` Pierre Ossman
  0 siblings, 1 reply; 5+ messages in thread
From: OGAWA Hirofumi @ 2007-02-19 16:55 UTC (permalink / raw)
  To: Pierre Ossman; +Cc: LKML

Pierre Ossman <drzeus-list@drzeus.cx> writes:

> Hi,

Hi,

> I'm experiencing a rather odd behaviour with the character set
> conversion. If I mount a vfat fs with utf8 and then create a file
> with invalid utf-8 sequences, the file will briefly exist with these
> invalid sequences, then quickly convert to a stripped version.

Yes. utf8 support is broken, and it will fail to convert letter case
on many case. And it's why that is not recommended.

> I haven't found an easy way to catch the race, but if I have
> nautilus open it tends to catch it now and then (I get a file name
> with "<?>" replacing each bad byte).

`?' is meaning the character can't be converted.

> The race also seems to corrupt the in-memory state of the fs now and then. I
> managed to create a file where "ls" shows "?" for most fields. Data seemed to
> have made it to disk ok though (fsck didn't complain and a remount showed
> everything as it should be).

It seems readdir() was success, but stat() was failed. And the dosfsck
doesn't check filename correctly.

Those problem seem to be relateing to poor utf8 support...

> Third, there seems to be a problem with not all syscalls being subjected to the
> NLS transformation. Example:
>
> $ echo foo > baråäö.txt
> $ ls
> foo.txt
> $ echo foo > baråäö.txt
> bash: baråäö.txt: File exists
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Racy NLS behaviour in FAT (and possible other fs)
  2007-02-19 16:55 ` OGAWA Hirofumi
@ 2007-02-19 17:03   ` Pierre Ossman
  2007-02-19 17:32     ` OGAWA Hirofumi
  0 siblings, 1 reply; 5+ messages in thread
From: Pierre Ossman @ 2007-02-19 17:03 UTC (permalink / raw)
  To: OGAWA Hirofumi; +Cc: LKML

OGAWA Hirofumi wrote:
>> I'm experiencing a rather odd behaviour with the character set
>> conversion. If I mount a vfat fs with utf8 and then create a file
>> with invalid utf-8 sequences, the file will briefly exist with these
>> invalid sequences, then quickly convert to a stripped version.
>>     
>
> Yes. utf8 support is broken, and it will fail to convert letter case
> on many case. And it's why that is not recommended.
>
>   

Is there any ongoing work to fix this? UTF-8 is standard on more or less
every distribution these days.

Rgds

-- 
     -- Pierre Ossman

  Linux kernel, MMC maintainer        http://www.kernel.org
  PulseAudio, core developer          http://pulseaudio.org
  rdesktop, core developer          http://www.rdesktop.org


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Racy NLS behaviour in FAT (and possible other fs)
  2007-02-19 17:03   ` Pierre Ossman
@ 2007-02-19 17:32     ` OGAWA Hirofumi
  2007-02-20  6:43       ` Pierre Ossman
  0 siblings, 1 reply; 5+ messages in thread
From: OGAWA Hirofumi @ 2007-02-19 17:32 UTC (permalink / raw)
  To: Pierre Ossman; +Cc: LKML

Pierre Ossman <drzeus-list@drzeus.cx> writes:

> OGAWA Hirofumi wrote:
>>> I'm experiencing a rather odd behaviour with the character set
>>> conversion. If I mount a vfat fs with utf8 and then create a file
>>> with invalid utf-8 sequences, the file will briefly exist with these
>>> invalid sequences, then quickly convert to a stripped version.
>>
>> Yes. utf8 support is broken, and it will fail to convert letter case
>> on many case. And it's why that is not recommended.
>
> Is there any ongoing work to fix this? UTF-8 is standard on more or less
> every distribution these days.

Yes. But sorry, I don't have any plan and time to fix it now.

If you are using "iocharset=utf8" now, "codepage=cp???,iocharset=xxx,utf8"
might help a bit.
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Racy NLS behaviour in FAT (and possible other fs)
  2007-02-19 17:32     ` OGAWA Hirofumi
@ 2007-02-20  6:43       ` Pierre Ossman
  0 siblings, 0 replies; 5+ messages in thread
From: Pierre Ossman @ 2007-02-20  6:43 UTC (permalink / raw)
  To: OGAWA Hirofumi; +Cc: LKML

OGAWA Hirofumi wrote:
> Yes. But sorry, I don't have any plan and time to fix it now.
>   

I know the feeling. :)

I just wanted to know where things stand as I have little insight into
vfat development.

> If you are using "iocharset=utf8" now, "codepage=cp???,iocharset=xxx,utf8"
> might help a bit.
>   

These tests was with the "utf8" option.

Rgds

-- 
     -- Pierre Ossman

  Linux kernel, MMC maintainer        http://www.kernel.org
  PulseAudio, core developer          http://pulseaudio.org
  rdesktop, core developer          http://www.rdesktop.org


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2007-02-20  6:42 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-02-19 15:53 Racy NLS behaviour in FAT (and possible other fs) Pierre Ossman
2007-02-19 16:55 ` OGAWA Hirofumi
2007-02-19 17:03   ` Pierre Ossman
2007-02-19 17:32     ` OGAWA Hirofumi
2007-02-20  6:43       ` Pierre Ossman

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).