The commit 026d0d27c488 (ext4: reduce computation of overhead during resize) reduces the time taken to resize large bigalloc filesystems(reduces 3+ hours to milliseconds for a 64TB FS). This is a good candidate to cherry-pick to stable releases.
Kiselev, Oleg (1): ext4: reduce computation of overhead during resize
fs/ext4/resize.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-)
From: "Kiselev, Oleg" okiselev@amazon.com
commit 026d0d27c4882303e4b071ca6d996640cc2932c3 upstream.
This patch avoids doing an O(n**2)-complexity walk through every flex group. Instead, it uses the already computed overhead information for the newly allocated space, and simply adds it to the previously calculated overhead stored in the superblock. This drastically reduces the time taken to resize very large bigalloc filesystems (from 3+ hours for a 64TB fs down to milliseconds).
Signed-off-by: Oleg Kiselev okiselev@amazon.com Link: https://lore.kernel.org/r/CE4F359F-4779-45E6-B6A9-8D67FDFF5AE2@amazon.com Signed-off-by: Theodore Ts'o tytso@mit.edu Signed-off-by: Meena Shanmugam meenashanmugam@google.com --- fs/ext4/resize.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-)
diff --git a/fs/ext4/resize.c b/fs/ext4/resize.c index 51cebc1990eb..c606f16c94be 100644 --- a/fs/ext4/resize.c +++ b/fs/ext4/resize.c @@ -1360,6 +1360,17 @@ static int ext4_setup_new_descs(handle_t *handle, struct super_block *sb, return err; }
+static void ext4_add_overhead(struct super_block *sb, + const ext4_fsblk_t overhead) +{ + struct ext4_sb_info *sbi = EXT4_SB(sb); + struct ext4_super_block *es = sbi->s_es; + + sbi->s_overhead += overhead; + es->s_overhead_clusters = cpu_to_le32(sbi->s_overhead); + smp_wmb(); +} + /* * ext4_update_super() updates the super block so that the newly added * groups can be seen by the filesystem. @@ -1458,9 +1469,17 @@ static void ext4_update_super(struct super_block *sb, }
/* - * Update the fs overhead information + * Update the fs overhead information. + * + * For bigalloc, if the superblock already has a properly calculated + * overhead, update it with a value based on numbers already computed + * above for the newly allocated capacity. */ - ext4_calculate_overhead(sb); + if (ext4_has_feature_bigalloc(sb) && (sbi->s_overhead != 0)) + ext4_add_overhead(sb, + EXT4_NUM_B2C(sbi, blocks_count - free_blocks)); + else + ext4_calculate_overhead(sb); es->s_overhead_clusters = cpu_to_le32(sbi->s_overhead);
if (test_opt(sb, DEBUG))
Meena,
This patch implicitly relies on correctness of the overhead field in the superblock. In the older kernels, online resize neglected to update the overhead field. Ted (or Lucas?) fixed that, but I don't know if that patch got cherry-picked for the older stable releases. I don't think my patch will work right without those fixes.
On 5/1/23, 10:26 PM, "Meena Shanmugam" <meenashanmugam@google.com mailto:meenashanmugam@google.com> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
The commit 026d0d27c488 (ext4: reduce computation of overhead during resize) reduces the time taken to resize large bigalloc filesystems(reduces 3+ hours to milliseconds for a 64TB FS). This is a good candidate to cherry-pick to stable releases.
Kiselev, Oleg (1): ext4: reduce computation of overhead during resize
fs/ext4/resize.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-)
-- 2.40.1.495.gc816e09b53d-goog
linux-stable-mirror@lists.linaro.org
-
Kiselev, Oleg -
Meena Shanmugam