Linux-Fsdevel Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Tetsuhiro Kohada <kohada.t2@gmail.com>
To: Sungjong Seo <sj1557.seo@samsung.com>
Cc: kohada.tetsuhiro@dc.mitsubishielectric.co.jp,
	mori.takahiro@ab.mitsubishielectric.co.jp,
	motai.hirotaka@aj.mitsubishielectric.co.jp,
	'Namjae Jeon' <namjae.jeon@samsung.com>,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3] exfat: remove EXFAT_SB_DIRTY flag
Date: Wed, 12 Aug 2020 18:19:32 +0900	[thread overview]
Message-ID: <490837eb-6765-c7be-bb80-b30fe34adb55@gmail.com> (raw)
In-Reply-To: <000301d66dac$07b9fc00$172df400$@samsung.com>

>>
>> When should VOL_DIRTY be cleared?
>>
>> The current behavior is ...
>>
>> Case of  mkdir, rmdir, rename:
>>     - set VOL_DIRTY before operation
>>     - set VOL_CLEAN after operating.
>> In async mode, it is actually written to the media after 30 seconds.
>>
>> Case of  cp, touch:
>>     - set VOL_DIRTY before operation
>>     - however, VOL_CLEAN is not called in this context.
>> VOL_CLEAN will call by sync_fs or unmount.
>>
>> I added VOL_CLEAN in last of __exfat_write_inode() and exfat_map_cluster().
>> As a result, VOL_DIRTY is cleared with cp and touch.
>> However, when copying a many files ...
>>    - Async mode: VOL_DIRTY is written to the media twice every 30 seconds.
>>    - Sync mode: Of course,  VOL_DIRTY and VOL_CLEAN to the media for each
>> file.
>>
>> Frequent writing VOL_DIRTY and VOL_CLEAN  increases the risk of boot-
>> sector curruption.
>> If the boot-sector corrupted, it causes the following serious problems  on
>> some OSs.
>>    - misjudge as unformatted
>>    - can't judge as exfat
>>    - can't repair
>>
>> I want to minimize boot sector writes, to reduce these risk.
>>
>> I looked vfat/udf implementation, which manages similar dirty information
>> on linux, and found that they ware mark-dirty at mount and cleared at
>> unmount.
>>
>> Here are some ways to clear VOL_DIRTY.
>>
>> (A) VOL_CLEAN after every write operation.
>>     :-) Ejectable at any time after a write operation.
>>     :-( Many times write to Boot-sector.
>>
>> (B) dirty at mount, clear at unmount (same as vfat/udf)
>>     :-) Write to boot-sector twice.
>>     :-( It remains dirty unless unmounted.
>>     :-( Write to boot-sector even if there is no write operation.
>>
>> (C) dirty on first write operation, clear on unmount
>>     :-) Writing to boot-sector is minimal.
>>     :-) Will not write to the boot-sector if there is no write operation.
>>     :-( It remains dirty unless unmounted.
>>
>> (D) dirty on first write operation,  clear on sync-fs/unmount
>>    :-) Writing to boot-sector can be reduced.
>>    :-) Will not write to the boot-sector if there is no write operation.
>>    :-) sync-fs makes it clean and ejectable immidiately.
>>    :-( It remains dirty unless sync-fs or unmount.
>>    :-( Frequent sync-fs will  increases writes to boot-sector.
>>
>> I think it should be (C) or(D).
>> What do you think?
>>
> 
> First of all, I'm sorry for the late reply.
> And thank you for the suggestion.

Thanks for thinking on this complicated issue.


> Most of the NAND flash devices and HDDs have wear leveling and bad sector replacement algorithms applied.
> So I think that the life of the boot sector will not be exhausted first.

I'm not too worried about the life of the boot-sector.
I'm worried about write failures caused by external factors.
(power failure/system down/vibration/etc. during writing)
They rarely occur on SD cards, but occur on many HDDs, some SSDs and USB storages by 0.1% or more.
Especially with AFT-HDD, not only boot-sector but also the following multiple sectors become unreadable.
It is not possible to completely solve this problem, as long as writing to the boot-sector.
(I think it's a exFAT's specification defect)
The only effective way to reduce this problem is to reduce writes to the boot-sector.


> Currently the volume dirty/clean policy of exfat-fs is not perfect,

Thank you for sharing the problem with you.


> but I think it behaves similarly to the policy of MS Windows.

On Windows10, the dirty flag is cleared after more than 15 seconds after all write operations are completed.
(dirty-flag is never updated during the write operation continues)


> Therefore,
> I think code improvements should be made to reduce volume flag records while maintaining the current policy.

Current policy is inconsistent.
As I wrote last mail, the problem with the current implementation is that the dirty-flag may not be cleared
after the write operation.(even if sync is enabled or disabled)
Because, some write operations clear the dirty-flag but some don't clear.
Unmount or sync command is the only way to ensure that the dirty-flag is cleared.
This has no effect on clearing the dirty-flag after a write operations, it only increases risk of destroying
the boot-sector.
  - Clear the dirty-flag after every write operation.
  - Never clear the dirty-flag after every write operation.
Unless unified to either one,  I think that sync policy cannot be consistent.

How do you think?


BR
---
etsuhiro Kohada <kohada.t2@gmail.com>


  reply	other threads:[~2020-08-12  9:19 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CGME20200616021816epcas1p44b0833f14bbad0e25cc0efb27fb2ebd3@epcas1p4.samsung.com>
2020-06-16  2:18 ` Tetsuhiro Kohada
2020-06-16 23:55   ` Namjae Jeon
2020-06-17  7:20   ` Sungjong Seo
2020-06-17  8:41     ` Namjae Jeon
2020-06-18  8:36     ` Tetsuhiro Kohada
2020-06-18 13:11       ` Sungjong Seo
2020-06-19  4:22         ` Tetsuhiro Kohada
2020-07-10  7:36         ` Tetsuhiro Kohada
2020-08-08 17:47           ` Sungjong Seo
2020-08-12  9:19             ` Tetsuhiro Kohada [this message]
2020-08-13  4:03               ` Namjae Jeon
2020-08-18  1:20                 ` Tetsuhiro Kohada
2020-06-16  6:16 Markus Elfring
2020-06-16 14:45 ` Matthew Wilcox

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=490837eb-6765-c7be-bb80-b30fe34adb55@gmail.com \
    --to=kohada.t2@gmail.com \
    --cc=kohada.tetsuhiro@dc.mitsubishielectric.co.jp \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mori.takahiro@ab.mitsubishielectric.co.jp \
    --cc=motai.hirotaka@aj.mitsubishielectric.co.jp \
    --cc=namjae.jeon@samsung.com \
    --cc=sj1557.seo@samsung.com \
    --subject='Re: [PATCH v3] exfat: remove EXFAT_SB_DIRTY flag' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).