From: Ard Biesheuvel ardb@kernel.org
On x86, the ordinary, position dependent 'small' and 'kernel' code models only support placement of the executable in 32-bit addressable memory, due to the use of 32-bit signed immediates to generate references to global variables. For the kernel, this implies that all global variables must reside in the top 2 GiB of the kernel virtual address space, where the implicit address bits 63:32 are equal to sign bit 31.
This means the kernel code model is not suitable for other bare metal executables such as the kexec purgatory, which can be placed arbitrarily in the physical address space, where its address may no longer be representable as a sign extended 32-bit quantity. For this reason, commit
e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
switched to the 'large' code model, which uses 64-bit immediates for all symbol references, including function calls, in order to avoid relying on any assumptions regarding proximity of symbols in the final executable.
The large code model is rarely used, clunky and the least likely to operate in a similar fashion when comparing GCC and Clang, so it is best avoided. This is especially true now that Clang 18 has started to emit executable code in two separate sections (.text and .ltext), which triggers an issue in the kexec loading code at runtime.
Instead, use the position independent small code model, which makes no assumptions about placement but only about proximity, where all referenced symbols must be within -/+ 2 GiB, i.e., in range for a RIP-relative reference. Use hidden visibility to suppress the use of a GOT, which carries absolute addresses that are not covered by static ELF relocations, and is therefore incompatible with the kexec loader's relocation logic.
Cc: Nathan Chancellor nathan@kernel.org Cc: Nick Desaulniers ndesaulniers@google.com Cc: Bill Wendling morbo@google.com Cc: Justin Stitt justinstitt@google.com Cc: Song Liu song@kernel.org Cc: Ricardo Ribalda ribalda@kernel.org Cc: Fangrui Song maskray@google.com Cc: Arthur Eubanks aeubanks@google.com Link: https://lore.kernel.org/all/20240417-x86-fix-kexec-with-llvm-18-v1-0-5383121... Cc: stable@vger.kernel.org Signed-off-by: Ard Biesheuvel ardb@kernel.org --- arch/x86/purgatory/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/x86/purgatory/Makefile b/arch/x86/purgatory/Makefile index bc31863c5ee6..a18591f6e6d9 100644 --- a/arch/x86/purgatory/Makefile +++ b/arch/x86/purgatory/Makefile @@ -42,7 +42,8 @@ KCOV_INSTRUMENT := n # make up the standalone purgatory.ro
PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel -PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS := -mcmodel=small -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS += -fpic -fvisibility=hidden PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN) -DDISABLE_BRANCH_PROFILING PURGATORY_CFLAGS += -fno-stack-protector
On Thu, Apr 18, 2024 at 1:17 PM Ard Biesheuvel ardb+git@google.com wrote:
From: Ard Biesheuvel ardb@kernel.org
On x86, the ordinary, position dependent 'small' and 'kernel' code models only support placement of the executable in 32-bit addressable memory, due to the use of 32-bit signed immediates to generate references to global variables. For the kernel, this implies that all global variables must reside in the top 2 GiB of the kernel virtual address space, where the implicit address bits 63:32 are equal to sign bit 31.
This means the kernel code model is not suitable for other bare metal executables such as the kexec purgatory, which can be placed arbitrarily in the physical address space, where its address may no longer be representable as a sign extended 32-bit quantity. For this reason, commit
e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
switched to the 'large' code model, which uses 64-bit immediates for all symbol references, including function calls, in order to avoid relying on any assumptions regarding proximity of symbols in the final executable.
The large code model is rarely used, clunky and the least likely to operate in a similar fashion when comparing GCC and Clang, so it is best avoided. This is especially true now that Clang 18 has started to emit executable code in two separate sections (.text and .ltext), which triggers an issue in the kexec loading code at runtime.
Instead, use the position independent small code model, which makes no assumptions about placement but only about proximity, where all referenced symbols must be within -/+ 2 GiB, i.e., in range for a RIP-relative reference. Use hidden visibility to suppress the use of a GOT, which carries absolute addresses that are not covered by static ELF relocations, and is therefore incompatible with the kexec loader's relocation logic.
Cc: Nathan Chancellor nathan@kernel.org Cc: Nick Desaulniers ndesaulniers@google.com
Thanks Ard!
Acked-by: Nick Desaulniers ndesaulniers@google.com Reported-by: ns 0n-s@users.noreply.github.com Closes: https://github.com/ClangBuiltLinux/linux/issues/2016 Fixes: e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
(I don't have a kexec setup ready to go; maybe someone that does can help test it.)
Cc: Bill Wendling morbo@google.com Cc: Justin Stitt justinstitt@google.com Cc: Song Liu song@kernel.org Cc: Ricardo Ribalda ribalda@kernel.org Cc: Fangrui Song maskray@google.com Cc: Arthur Eubanks aeubanks@google.com Link: https://lore.kernel.org/all/20240417-x86-fix-kexec-with-llvm-18-v1-0-5383121... Cc: stable@vger.kernel.org Signed-off-by: Ard Biesheuvel ardb@kernel.org
arch/x86/purgatory/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/x86/purgatory/Makefile b/arch/x86/purgatory/Makefile index bc31863c5ee6..a18591f6e6d9 100644 --- a/arch/x86/purgatory/Makefile +++ b/arch/x86/purgatory/Makefile @@ -42,7 +42,8 @@ KCOV_INSTRUMENT := n # make up the standalone purgatory.ro
PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel -PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS := -mcmodel=small -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS += -fpic -fvisibility=hidden PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN) -DDISABLE_BRANCH_PROFILING PURGATORY_CFLAGS += -fno-stack-protector
-- 2.44.0.769.g3c40516874-goog
On 2024-04-18, Nick Desaulniers wrote:
On Thu, Apr 18, 2024 at 1:17 PM Ard Biesheuvel ardb+git@google.com wrote:
From: Ard Biesheuvel ardb@kernel.org
On x86, the ordinary, position dependent 'small' and 'kernel' code models only support placement of the executable in 32-bit addressable memory, due to the use of 32-bit signed immediates to generate references to global variables. For the kernel, this implies that all global variables must reside in the top 2 GiB of the kernel virtual address space, where the implicit address bits 63:32 are equal to sign bit 31.
This means the kernel code model is not suitable for other bare metal executables such as the kexec purgatory, which can be placed arbitrarily in the physical address space, where its address may no longer be representable as a sign extended 32-bit quantity. For this reason, commit
e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
switched to the 'large' code model, which uses 64-bit immediates for all symbol references, including function calls, in order to avoid relying on any assumptions regarding proximity of symbols in the final executable.
The large code model is rarely used, clunky and the least likely to operate in a similar fashion when comparing GCC and Clang, so it is best avoided. This is especially true now that Clang 18 has started to emit executable code in two separate sections (.text and .ltext), which triggers an issue in the kexec loading code at runtime.
Instead, use the position independent small code model, which makes no assumptions about placement but only about proximity, where all referenced symbols must be within -/+ 2 GiB, i.e., in range for a RIP-relative reference. Use hidden visibility to suppress the use of a GOT, which carries absolute addresses that are not covered by static ELF relocations, and is therefore incompatible with the kexec loader's relocation logic.
Cc: Nathan Chancellor nathan@kernel.org Cc: Nick Desaulniers ndesaulniers@google.com
Thanks Ard!
Acked-by: Nick Desaulniers ndesaulniers@google.com Reported-by: ns 0n-s@users.noreply.github.com Closes: https://github.com/ClangBuiltLinux/linux/issues/2016 Fixes: e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
(I don't have a kexec setup ready to go; maybe someone that does can help test it.)
LGTM.
Position-dependent small code model may generate R_X86_64_32S relocations with a range of [0,2GiB) (the negative half cannot be used). Position-independent small code model with hidden visibility will generate R_X86_64_PC32 and can typically be quite larger than 2G without hitting an overflow issue.
(I have some notes about R_X86_64_32S at https://maskray.me/blog/2023-05-14-relocation-overflow-and-code-models#x86-6...)
Reviewed-by: Fangrui Song maskray@google.com
Cc: Bill Wendling morbo@google.com Cc: Justin Stitt justinstitt@google.com Cc: Song Liu song@kernel.org Cc: Ricardo Ribalda ribalda@kernel.org Cc: Fangrui Song maskray@google.com Cc: Arthur Eubanks aeubanks@google.com Link: https://lore.kernel.org/all/20240417-x86-fix-kexec-with-llvm-18-v1-0-5383121... Cc: stable@vger.kernel.org Signed-off-by: Ard Biesheuvel ardb@kernel.org
arch/x86/purgatory/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/x86/purgatory/Makefile b/arch/x86/purgatory/Makefile index bc31863c5ee6..a18591f6e6d9 100644 --- a/arch/x86/purgatory/Makefile +++ b/arch/x86/purgatory/Makefile @@ -42,7 +42,8 @@ KCOV_INSTRUMENT := n # make up the standalone purgatory.ro
PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel -PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS := -mcmodel=small -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS += -fpic -fvisibility=hidden PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN) -DDISABLE_BRANCH_PROFILING PURGATORY_CFLAGS += -fno-stack-protector
-- 2.44.0.769.g3c40516874-goog
-- Thanks, ~Nick Desaulniers
On Thu, Apr 18, 2024 at 01:36:22PM -0700, Nick Desaulniers wrote:
Reported-by: ns 0n-s@users.noreply.github.com
I don't know what the goal with this tag is but this'll keep bouncing:
0n-s@users.noreply.github.com: Host or domain name not found. Name service error for name=users.noreply.github.com type=AAAA: Host not found
Maybe use a non-void email...?
On Sat, 20 Apr 2024 at 17:30, Borislav Petkov bp@alien8.de wrote:
On Thu, Apr 18, 2024 at 01:36:22PM -0700, Nick Desaulniers wrote:
Reported-by: ns 0n-s@users.noreply.github.com
I don't know what the goal with this tag is but this'll keep bouncing:
0n-s@users.noreply.github.com: Host or domain name not found. Name service error for name=users.noreply.github.com type=AAAA: Host not found
Maybe use a non-void email...?
Let's just drop that.
If people cannot be bothered to use their real name, I don't think they care about being credited either.
On Thu, Apr 18, 2024 at 10:17:06PM +0200, Ard Biesheuvel wrote:
From: Ard Biesheuvel ardb@kernel.org
On x86, the ordinary, position dependent 'small' and 'kernel' code models only support placement of the executable in 32-bit addressable memory, due to the use of 32-bit signed immediates to generate references to global variables. For the kernel, this implies that all global variables must reside in the top 2 GiB of the kernel virtual address space, where the implicit address bits 63:32 are equal to sign bit 31.
This means the kernel code model is not suitable for other bare metal executables such as the kexec purgatory, which can be placed arbitrarily in the physical address space, where its address may no longer be representable as a sign extended 32-bit quantity. For this reason, commit
e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
switched to the 'large' code model, which uses 64-bit immediates for all symbol references, including function calls, in order to avoid relying on any assumptions regarding proximity of symbols in the final executable.
The large code model is rarely used, clunky and the least likely to operate in a similar fashion when comparing GCC and Clang, so it is best avoided. This is especially true now that Clang 18 has started to emit executable code in two separate sections (.text and .ltext), which triggers an issue in the kexec loading code at runtime.
Instead, use the position independent small code model, which makes no assumptions about placement but only about proximity, where all referenced symbols must be within -/+ 2 GiB, i.e., in range for a RIP-relative reference. Use hidden visibility to suppress the use of a GOT, which carries absolute addresses that are not covered by static ELF relocations, and is therefore incompatible with the kexec loader's relocation logic.
Cc: Nathan Chancellor nathan@kernel.org Cc: Nick Desaulniers ndesaulniers@google.com Cc: Bill Wendling morbo@google.com Cc: Justin Stitt justinstitt@google.com Cc: Song Liu song@kernel.org Cc: Ricardo Ribalda ribalda@kernel.org Cc: Fangrui Song maskray@google.com Cc: Arthur Eubanks aeubanks@google.com Link: https://lore.kernel.org/all/20240417-x86-fix-kexec-with-llvm-18-v1-0-5383121... Cc: stable@vger.kernel.org Signed-off-by: Ard Biesheuvel ardb@kernel.org
Reviewed-by: Nathan Chancellor nathan@kernel.org Tested-by: Nathan Chancellor nathan@kernel.org
This resolves the warning and relocation overflow error that I see with LLVM 18.1.4 (the relocation error was fixed in LLVM but dropping -mcmodel=large resolves it for toolchains without that fix).
arch/x86/purgatory/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/x86/purgatory/Makefile b/arch/x86/purgatory/Makefile index bc31863c5ee6..a18591f6e6d9 100644 --- a/arch/x86/purgatory/Makefile +++ b/arch/x86/purgatory/Makefile @@ -42,7 +42,8 @@ KCOV_INSTRUMENT := n # make up the standalone purgatory.ro PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel -PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS := -mcmodel=small -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS += -fpic -fvisibility=hidden PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN) -DDISABLE_BRANCH_PROFILING PURGATORY_CFLAGS += -fno-stack-protector -- 2.44.0.769.g3c40516874-goog
On 18. 04. 24, 22:17, Ard Biesheuvel wrote:
From: Ard Biesheuvel ardb@kernel.org
On x86, the ordinary, position dependent 'small' and 'kernel' code models only support placement of the executable in 32-bit addressable memory, due to the use of 32-bit signed immediates to generate references to global variables. For the kernel, this implies that all global variables must reside in the top 2 GiB of the kernel virtual address space, where the implicit address bits 63:32 are equal to sign bit 31.
This means the kernel code model is not suitable for other bare metal executables such as the kexec purgatory, which can be placed arbitrarily in the physical address space, where its address may no longer be representable as a sign extended 32-bit quantity. For this reason, commit
e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
switched to the 'large' code model, which uses 64-bit immediates for all symbol references, including function calls, in order to avoid relying on any assumptions regarding proximity of symbols in the final executable.
The large code model is rarely used, clunky and the least likely to operate in a similar fashion when comparing GCC and Clang, so it is best avoided. This is especially true now that Clang 18 has started to emit executable code in two separate sections (.text and .ltext), which triggers an issue in the kexec loading code at runtime.
Instead, use the position independent small code model, which makes no assumptions about placement but only about proximity, where all referenced symbols must be within -/+ 2 GiB, i.e., in range for a RIP-relative reference. Use hidden visibility to suppress the use of a GOT, which carries absolute addresses that are not covered by static ELF relocations, and is therefore incompatible with the kexec loader's relocation logic.
FWIW:
Fixes: https://bugzilla.suse.com/show_bug.cgi?id=1211853
thanks,
On Fri, Apr 19, 2024 at 01:35:44PM +0200, Jiri Slaby wrote:
On 18. 04. 24, 22:17, Ard Biesheuvel wrote:
From: Ard Biesheuvel ardb@kernel.org
On x86, the ordinary, position dependent 'small' and 'kernel' code models only support placement of the executable in 32-bit addressable memory, due to the use of 32-bit signed immediates to generate references to global variables. For the kernel, this implies that all global variables must reside in the top 2 GiB of the kernel virtual address space, where the implicit address bits 63:32 are equal to sign bit 31.
This means the kernel code model is not suitable for other bare metal executables such as the kexec purgatory, which can be placed arbitrarily in the physical address space, where its address may no longer be representable as a sign extended 32-bit quantity. For this reason, commit
e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
switched to the 'large' code model, which uses 64-bit immediates for all symbol references, including function calls, in order to avoid relying on any assumptions regarding proximity of symbols in the final executable.
The large code model is rarely used, clunky and the least likely to operate in a similar fashion when comparing GCC and Clang, so it is best avoided. This is especially true now that Clang 18 has started to emit executable code in two separate sections (.text and .ltext), which triggers an issue in the kexec loading code at runtime.
Instead, use the position independent small code model, which makes no assumptions about placement but only about proximity, where all referenced symbols must be within -/+ 2 GiB, i.e., in range for a RIP-relative reference. Use hidden visibility to suppress the use of a GOT, which carries absolute addresses that are not covered by static ELF relocations, and is therefore incompatible with the kexec loader's relocation logic.
FWIW:
Interesting. I thought gcc doesn't have problems here yet and was talking to Matz on Thu about it and it seems he's forgotten about his statement too that "you should simply stop using -mcmodel=large. Noone should use it." :-)
Ok then, lemme queue Ard's fix.
I guess we want it in stable too but as it is pretty much the case with toolchain fixes I'd prefer if we queued it the normal path for 6.10 and then backport it after it sees some testing and nothing breaks. And not rush it to Linus now.
Thx.
Hello,
On Sat, 20 Apr 2024, Borislav Petkov wrote:
Interesting. I thought gcc doesn't have problems here yet and was talking to Matz on Thu about it and it seems he's forgotten about his statement too that "you should simply stop using -mcmodel=large. Noone should use it." :-)
It may be so ingrained in my brain that I'm not _always_ saying it when talking about the large code model over a beer. And indeed I know of no particular problems with it vis GCC, but that doesn't mean it's a good idea to use :-)
So once again: "everyone should simply stop using -mcmodel=large. Noone should use it."
So the patch goes strictly into the direction of betterment of the universe. :)
Ciao, Michael.
On Mon, Apr 29, 2024 at 02:05:12PM +0200, Michael Matz wrote:
It may be so ingrained in my brain that I'm not _always_ saying it when talking about the large code model over a beer.
Doh, you should. This is what you start with! :-P
And indeed I know of no particular problems with it vis GCC, but that doesn't mean it's a good idea to use :-)
So once again: "everyone should simply stop using -mcmodel=large. Noone should use it."
So the patch goes strictly into the direction of betterment of the universe. :)
Yeah, it is already on its way to every kernel near you. And looka here:
$ git grep mcmodel=large arch/powerpc/Makefile:125: # 64bit relocation for this to work, hence -mcmodel=large. arch/powerpc/Makefile:126: KBUILD_CFLAGS_MODULE += -mcmodel=large arch/um/Makefile:34: KBUILD_CFLAGS += -mcmodel=large
x86 is all free of the large model now.
One less thing to worry about - gazillion more to go.
:-P
See ya on Thu.
Hi,
On 29. 04. 24, 14:05, Michael Matz wrote:
On Sat, 20 Apr 2024, Borislav Petkov wrote:
Interesting. I thought gcc doesn't have problems here yet and was talking to Matz on Thu about it and it seems he's forgotten about his statement too that "you should simply stop using -mcmodel=large. Noone should use it." :-)
It may be so ingrained in my brain that I'm not _always_ saying it when talking about the large code model over a beer. And indeed I know of no particular problems with it vis GCC,
Of course you do :). That bsc#1211853 I linked earlier. I.e. gcc-13 + -fstrict-flex-arrays=3 + -mcmodel=large + some asm() expecting __FILE__ to be constant (not true with the large model).
regards,
Hello,
On Tue, 30 Apr 2024, Jiri Slaby wrote:
Interesting. I thought gcc doesn't have problems here yet and was talking to Matz on Thu about it and it seems he's forgotten about his statement too that "you should simply stop using -mcmodel=large. Noone should use it." :-)
It may be so ingrained in my brain that I'm not _always_ saying it when talking about the large code model over a beer. And indeed I know of no particular problems with it vis GCC,
Of course you do :).
:-P
That bsc#1211853 I linked earlier. I.e. gcc-13 + -fstrict-flex-arrays=3 + -mcmodel=large + some asm() expecting __FILE__ to be constant (not true with the large model).
"asm() expecting $whatever" - clearly a user problem, not a GCC problem ;-)
Ciao, Michael.
linux-stable-mirror@lists.linaro.org
-
Ard Biesheuvel -
Ard Biesheuvel -
Borislav Petkov -
Fangrui Song -
Jiri Slaby -
Michael Matz -
Nathan Chancellor -
Nick Desaulniers