From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755824AbXD0NjR (ORCPT ); Fri, 27 Apr 2007 09:39:17 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1755831AbXD0NjR (ORCPT ); Fri, 27 Apr 2007 09:39:17 -0400 Received: from keetweej.vanheusden.com ([213.84.46.114]:41718 "EHLO keetweej.vanheusden.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755824AbXD0NjP (ORCPT ); Fri, 27 Apr 2007 09:39:15 -0400 Date: Fri, 27 Apr 2007 15:39:12 +0200 From: Folkert van Heusden To: linux-kernel@vger.kernel.org Cc: udovdh@xs4all.nl Subject: [2.6.21] possible circular locking dependency detected IN FILESYSTEM CODE (ext3) Message-ID: <20070427133912.GN1021@vanheusden.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Organization: www.unixexpert.nl X-Chameleon-Return-To: folkert@vanheusden.com X-Xfmail-Return-To: folkert@vanheusden.com X-Phonenumber: +31-6-41278122 X-URL: http://www.vanheusden.com/ X-PGP-KeyID: 1F28D8AE X-GPG-fingerprint: AC89 09CE 41F2 00B4 FCF2 B174 3019 0E8C 1F28 D8AE X-Key: http://pgp.surfnet.nl:11371/pks/lookup?op=get&search=0x1F28D8AE Read-Receipt-To: Reply-By: Sat Apr 28 13:36:06 CEST 2007 X-Message-Flag: MultiTail - tail on steroids User-Agent: Mutt/1.5.13 (2006-08-11) Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Hi, Running 2.6.21 on a P4 with HT and 2GB of ram. One filesystem which is ext3. After 2+ hours of uptime I got the following circular locking dependency in my dmesg output: [ 7429.853775] ======================================================= [ 7429.853884] [ INFO: possible circular locking dependency detected ] [ 7429.853939] 2.6.21 #1 [ 7429.854005] ------------------------------------------------------- [ 7429.854058] moo/5453 is trying to acquire lock: [ 7429.854112] (tty_mutex){--..}, at: [] mutex_lock+0x8/0xa [ 7429.854286] [ 7429.854287] but task is already holding lock: [ 7429.854413] (&s->s_dquot.dqptr_sem){----}, at: [] dquot_alloc_space+0x50/0x189 [ 7429.854423] [ 7429.854424] which lock already depends on the new lock. [ 7429.854426] [ 7429.854427] [ 7429.854428] the existing dependency chain (in reverse order) is: [ 7429.854431] [ 7429.854432] -> #4 (&s->s_dquot.dqptr_sem){----}: [ 7429.854437] [] check_prev_add+0x15b/0x281 [ 7429.854445] [] check_prevs_add+0x8b/0xe8 [ 7429.854452] [] __lock_acquire+0x692/0xb81 [ 7429.854460] [] lock_acquire+0x62/0x81 [ 7429.854468] [] down_read+0x2b/0x3d [ 7429.854477] [] dquot_alloc_space+0x50/0x189 [ 7429.854490] [] ext3_new_blocks+0x44b/0x5a2 [ 7429.854497] [] ext3_alloc_blocks+0x40/0xdf [ 7429.854505] [] ext3_alloc_branch+0x50/0x21b [ 7429.854512] [] ext3_get_blocks_handle+0x1b8/0x367 [ 7429.854519] [] ext3_getblk+0x97/0x228 [ 7429.854530] [] ext3_bread+0x1a/0x78 [ 7429.854536] [] ext3_mkdir+0xf4/0x270 [ 7429.854545] [] vfs_mkdir+0xb3/0x161 [ 7429.854553] [] sys_mkdirat+0x8c/0xc4 [ 7429.854560] [] sys_mkdir+0x20/0x22 [ 7429.854570] [] syscall_call+0x7/0xb [ 7429.854578] [] 0xffffffff [ 7429.854600] [ 7429.854600] -> #3 (&ei->truncate_mutex){--..}: [ 7429.854608] [] check_prev_add+0x15b/0x281 [ 7429.854616] [] check_prevs_add+0x8b/0xe8 [ 7429.854623] [] __lock_acquire+0x692/0xb81 [ 7429.854630] [] lock_acquire+0x62/0x81 [ 7429.854637] [] __mutex_lock_slowpath+0x75/0x28c [ 7429.854647] [] mutex_lock+0x8/0xa [ 7429.854654] [] ext3_truncate+0x170/0x468 [ 7429.854661] [] vmtruncate+0xa6/0x116 [ 7429.854669] [] inode_setattr+0x145/0x16c [ 7429.854678] [] ext3_setattr+0x150/0x22f [ 7429.854690] [] notify_change+0x352/0x386 [ 7429.854697] [] do_truncate+0x52/0x75 [ 7429.854705] [] may_open+0x1ec/0x231 [ 7429.854713] [] open_namei+0xda/0x59b [ 7429.854720] [] do_filp_open+0x2c/0x53 [ 7429.854731] [] do_sys_open+0x52/0xd8 [ 7429.854738] [] sys_open+0x1c/0x1e [ 7429.854745] [] syscall_call+0x7/0xb [ 7429.854752] [] 0xffffffff [ 7429.854760] [ 7429.854761] -> #2 (&inode->i_alloc_sem){--..}: [ 7429.854770] [] check_prev_add+0x15b/0x281 [ 7429.854777] [] check_prevs_add+0x8b/0xe8 [ 7429.854784] [] __lock_acquire+0x692/0xb81 [ 7429.854792] [] lock_acquire+0x62/0x81 [ 7429.854798] [] down_write+0x2b/0x45 [ 7429.854808] [] notify_change+0x2d9/0x386 [ 7429.854818] [] do_truncate+0x52/0x75 [ 7429.854826] [] may_open+0x1ec/0x231 [ 7429.854834] [] open_namei+0xda/0x59b [ 7429.854842] [] do_filp_open+0x2c/0x53 [ 7429.854850] [] do_sys_open+0x52/0xd8 [ 7429.854860] [] sys_open+0x1c/0x1e [ 7429.854867] [] syscall_call+0x7/0xb [ 7429.854874] [] 0xffffffff [ 7429.854882] [ 7429.854883] -> #1 (&sysfs_inode_imutex_key){--..}: [ 7429.854887] [] check_prev_add+0x15b/0x281 [ 7429.854895] [] check_prevs_add+0x8b/0xe8 [ 7429.854902] [] __lock_acquire+0x692/0xb81 [ 7429.854910] [] lock_acquire+0x62/0x81 [ 7429.854917] [] __mutex_lock_slowpath+0x75/0x28c [ 7429.854924] [] mutex_lock+0x8/0xa [ 7429.854934] [] remove_dir+0x2c/0xf1 [ 7429.854942] [] sysfs_remove_subdir+0x8/0xa [ 7429.854950] [] sysfs_remove_group+0x67/0x78 [ 7429.854957] [] dpm_sysfs_remove+0x12/0x16 [ 7429.854966] [] device_pm_remove+0x21/0x62 [ 7429.854977] [] device_del+0x89/0x1dd [ 7429.854984] [] device_unregister+0xb/0x15 [ 7429.854992] [] device_destroy+0x8d/0x9a [ 7429.854999] [] vcs_remove_sysfs+0x1c/0x39 [ 7429.855007] [] con_close+0x5e/0x6b [ 7429.855017] [] release_dev+0x4c4/0x6ce [ 7429.855025] [] tty_release+0x12/0x1c [ 7429.855033] [] __fput+0x145/0x16e [ 7429.855040] [] fput+0x19/0x1b [ 7429.855046] [] filp_close+0x3c/0x75 [ 7429.855056] [] sys_close+0x69/0xb4 [ 7429.855064] [] syscall_call+0x7/0xb [ 7429.855070] [] 0xffffffff [ 7429.855081] [ 7429.855082] -> #0 (tty_mutex){--..}: [ 7429.855085] [] check_prev_add+0x34/0x281 [ 7429.855098] [] check_prevs_add+0x8b/0xe8 [ 7429.855105] [] __lock_acquire+0x692/0xb81 [ 7429.855112] [] lock_acquire+0x62/0x81 [ 7429.855120] [] __mutex_lock_slowpath+0x75/0x28c [ 7429.855127] [] mutex_lock+0x8/0xa [ 7429.855134] [] print_warning+0x8c/0x15d [ 7429.855144] [] dquot_alloc_space+0x184/0x189 [ 7429.855151] [] ext3_new_blocks+0x44b/0x5a2 [ 7429.855162] [] ext3_alloc_blocks+0x40/0xdf [ 7429.855169] [] ext3_alloc_branch+0x50/0x21b [ 7429.855177] [] ext3_get_blocks_handle+0x1b8/0x367 [ 7429.855184] [] ext3_get_block+0x78/0xe3 [ 7429.855195] [] __block_prepare_write+0x168/0x408 [ 7429.855202] [] block_prepare_write+0x28/0x3b [ 7429.855210] [] ext3_prepare_write+0xe3/0x17e [ 7429.855218] [] generic_file_buffered_write+0x1b1/0x611 [ 7429.855226] [] __generic_file_aio_write_nolock+0x2ae/0x5ad [ 7429.855239] [] generic_file_aio_write+0x58/0xc4 [ 7429.855246] [] ext3_file_write+0x2d/0xba [ 7429.855253] [] do_sync_write+0xc7/0x116 [ 7429.855260] [] vfs_write+0x158/0x15d [ 7429.855266] [] sys_write+0x3d/0x64 [ 7429.855275] [] syscall_call+0x7/0xb [ 7429.855282] [] 0xffffffff [ 7429.855290] [ 7429.855291] other info that might help us debug this: [ 7429.855293] [ 7429.855296] 3 locks held by moo/5453: [ 7429.855297] #0: (&inode->i_mutex){--..}, at: [] mutex_lock+0x8/0xa [ 7429.855305] #1: (&ei->truncate_mutex){--..}, at: [] mutex_lock+0x8/0xa [ 7429.855312] #2: (&s->s_dquot.dqptr_sem){----}, at: [] dquot_alloc_space+0x50/0x189 [ 7429.855323] [ 7429.855324] stack backtrace: [ 7429.855326] [] show_trace_log_lvl+0x1a/0x30 [ 7429.855331] [] show_trace+0x12/0x14 [ 7429.855335] [] dump_stack+0x16/0x18 [ 7429.855339] [] print_circular_bug_tail+0x6f/0x71 [ 7429.855343] [] check_prev_add+0x34/0x281 [ 7429.855347] [] check_prevs_add+0x8b/0xe8 [ 7429.855350] [] __lock_acquire+0x692/0xb81 [ 7429.855355] [] lock_acquire+0x62/0x81 [ 7429.855362] [] __mutex_lock_slowpath+0x75/0x28c [ 7429.855366] [] mutex_lock+0x8/0xa [ 7429.855370] [] print_warning+0x8c/0x15d [ 7429.855374] [] dquot_alloc_space+0x184/0x189 [ 7429.855378] [] ext3_new_blocks+0x44b/0x5a2 [ 7429.855382] [] ext3_alloc_blocks+0x40/0xdf [ 7429.855386] [] ext3_alloc_branch+0x50/0x21b [ 7429.855007] [] con_close+0x5e/0x6b [ 7429.855017] [] release_dev+0x4c4/0x6ce [ 7429.855025] [] tty_release+0x12/0x1c [ 7429.855033] [] __fput+0x145/0x16e [ 7429.855040] [] fput+0x19/0x1b [ 7429.855046] [] filp_close+0x3c/0x75 [ 7429.855056] [] sys_close+0x69/0xb4 [ 7429.855064] [] syscall_call+0x7/0xb [ 7429.855070] [] 0xffffffff [ 7429.855081] [ 7429.855082] -> #0 (tty_mutex){--..}: [ 7429.855085] [] check_prev_add+0x34/0x281 [ 7429.855098] [] check_prevs_add+0x8b/0xe8 [ 7429.855105] [] __lock_acquire+0x692/0xb81 [ 7429.855112] [] lock_acquire+0x62/0x81 [ 7429.855120] [] __mutex_lock_slowpath+0x75/0x28c [ 7429.855127] [] mutex_lock+0x8/0xa [ 7429.855134] [] print_warning+0x8c/0x15d [ 7429.855144] [] dquot_alloc_space+0x184/0x189 [ 7429.855151] [] ext3_new_blocks+0x44b/0x5a2 [ 7429.855162] [] ext3_alloc_blocks+0x40/0xdf [ 7429.855169] [] ext3_alloc_branch+0x50/0x21b [ 7429.855177] [] ext3_get_blocks_handle+0x1b8/0x367 [ 7429.855184] [] ext3_get_block+0x78/0xe3 [ 7429.855195] [] __block_prepare_write+0x168/0x408 [ 7429.855202] [] block_prepare_write+0x28/0x3b [ 7429.855210] [] ext3_prepare_write+0xe3/0x17e [ 7429.855218] [] generic_file_buffered_write+0x1b1/0x611 [ 7429.855226] [] __generic_file_aio_write_nolock+0x2ae/0x5ad [ 7429.855239] [] generic_file_aio_write+0x58/0xc4 [ 7429.855246] [] ext3_file_write+0x2d/0xba [ 7429.855253] [] do_sync_write+0xc7/0x116 [ 7429.855260] [] vfs_write+0x158/0x15d [ 7429.855266] [] sys_write+0x3d/0x64 [ 7429.855275] [] syscall_call+0x7/0xb [ 7429.855282] [] 0xffffffff [ 7429.855290] [ 7429.855291] other info that might help us debug this: [ 7429.855293] [ 7429.855296] 3 locks held by moo/5453: [ 7429.855297] #0: (&inode->i_mutex){--..}, at: [] mutex_lock+0x8/0xa [ 7429.855305] #1: (&ei->truncate_mutex){--..}, at: [] mutex_lock+0x8/0xa [ 7429.855312] #2: (&s->s_dquot.dqptr_sem){----}, at: [] dquot_alloc_space+0x50/0x189 [ 7429.855323] [ 7429.855324] stack backtrace: [ 7429.855326] [] show_trace_log_lvl+0x1a/0x30 [ 7429.855331] [] show_trace+0x12/0x14 [ 7429.855335] [] dump_stack+0x16/0x18 [ 7429.855339] [] print_circular_bug_tail+0x6f/0x71 [ 7429.855343] [] check_prev_add+0x34/0x281 [ 7429.855347] [] check_prevs_add+0x8b/0xe8 [ 7429.855350] [] __lock_acquire+0x692/0xb81 [ 7429.855355] [] lock_acquire+0x62/0x81 [ 7429.855362] [] __mutex_lock_slowpath+0x75/0x28c [ 7429.855366] [] mutex_lock+0x8/0xa [ 7429.855370] [] print_warning+0x8c/0x15d [ 7429.855374] [] dquot_alloc_space+0x184/0x189 [ 7429.855378] [] ext3_new_blocks+0x44b/0x5a2 [ 7429.855382] [] ext3_alloc_blocks+0x40/0xdf [ 7429.855386] [] ext3_alloc_branch+0x50/0x21b [ 7429.855389] [] ext3_get_blocks_handle+0x1b8/0x367 [ 7429.855393] [] ext3_get_block+0x78/0xe3 [ 7429.855400] [] __block_prepare_write+0x168/0x408 [ 7429.855405] [] block_prepare_write+0x28/0x3b [ 7429.855409] [] ext3_prepare_write+0xe3/0x17e [ 7429.855412] [] generic_file_buffered_write+0x1b1/0x611 [ 7429.855417] [] __generic_file_aio_write_nolock+0x2ae/0x5ad [ 7429.855422] [] generic_file_aio_write+0x58/0xc4 [ 7429.855426] [] ext3_file_write+0x2d/0xba [ 7429.855430] [] do_sync_write+0xc7/0x116 [ 7429.855433] [] vfs_write+0x158/0x15d [ 7429.855440] [] sys_write+0x3d/0x64 [ 7429.855445] [] syscall_call+0x7/0xb [ 7429.855449] ======================= Folkert van Heusden -- www.biglumber.com <- site where one can exchange PGP key signatures ---------------------------------------------------------------------- Phone: +31-6-41278122, PGP-key: 1F28D8AE, www.vanheusden.com