Hi,
I noticed an ERR_PTR dereference issue in expand_files() on kernel 6.12.43 when allocating large file descriptor tables. The issue occurs when alloc_fdtable() returns ERR_PTR(-EMFILE) for large nr input, but expand_fdtable() is not properly checking these error returns. dup_fd() seems also have the issue, missing proper ERR_PTR handling.
The ERR_PTR return was introduced by d4f9351243c1 ("fs: Prevent file descriptor table allocations exceeding INT_MAX") which adds INT_MAX limit check in alloc_fdtable().
I was able to trigger this with the unshare_test selftest:
[ 40.283906] BUG: unable to handle page fault for address: ffffffffffffffe8 ... [ 40.287436] RIP: 0010:expand_files+0x7e/0x1c0 ... [ 40.366211] Kernel panic - not syncing: Fatal exception
Looking at the upstream kernel, this can be addressed by Al Viro's fdtable series [1], which added the ERR_PTR handling in this code path. Perhaps backporting this series, especially 1d3b4be ("alloc_fdtable(): change calling conventions.") would help resolve the issue.
Thanks for all the work on stable tree.
Best, Nathan Gao
[1] https://lore.kernel.org/all/20241007173912.GR4017910@ZenIV/
Signed-off-by: Nathan Gao zcgao@amazon.com