Can anyone help with my ARM + GCC + UCLIBC linking issue with crossdev?
Also posted to Gentoo Forums here: http://forums.gentoo.org/viewtopic-t-925012.html
Recently, I was assigned to a project that has executables developed using an old GCC with OABI. As a point of reference, here's a header output from readelf of an executable that runs just fine on the system:
ELF Header:
Magic: 7f 45 4c 46 01 01 01 61 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: ARM
ABI Version: 0
Type: EXEC (Executable file)
Machine: ARM
Version: 0x1
Entry point address: 0x9464
Start of program headers: 52 (bytes into file)
Start of section headers: 540956 (bytes into file)
Flags: 0x202, has entry point, GNU EABI, software FP
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 6
Size of section headers: 40 (bytes)
Number of section headers: 35
Section header string table index: 32
I creaed a cross-compiler using crossdev and the latest gcc/binutils/linux-headers/etc. and with EABI.
$ crossdev arm-softfloat-linux-uclibceabi
I quite happily began to populate my local folder with executables using that cross compiler only to later try the executable on my hardware and find out that I ended up with a segmentation fault. I realized, only through quite a bit of googling, that I really needed to have the old, legacy ABI for uclibc: OABI. My previous cross compiler was from circa 2005.
As another point of reference, my executables with eabi were producing headers from readelf that look a bit like this:
ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: ARM
Version: 0x1
Entry point address: 0x8130
Start of program headers: 52 (bytes into file)
Start of section headers: 21284 (bytes into file)
Flags: 0x5000002, has entry point, Version5 EABI
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 7
Size of section headers: 40 (bytes)
Number of section headers: 21
Section header string table index: 18
While the Machine is the same, the segmentation fault doesn't provide a way to execute the binary on the target.
After googling more, I found that there may be a way to produce some code with the eabi compiler for the legacy system. I was quite happy when I ran this command:
$ arm-softfloat-linux-uclibceabi-gcc -mabi=apcs-gnu -static -c -o /mnt/arm_uclibc/tmp/test /mnt/arm/tmp/test.c && readelf -h /mnt/arm_uclibc/tmp/test
And I ended up with:
ELF Header:
Magic: 7f 45 4c 46 01 01 01 61 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: ARM
ABI Version: 0
Type: REL (Relocatable file)
Machine: ARM
Version: 0x1
Entry point address: 0x0
Start of program headers: 0 (bytes into file)
Start of section headers: 248 (bytes into file)
Flags: 0x600, GNU EABI, software FP, VFP
Size of this header: 52 (bytes)
Size of program headers: 0 (bytes)
Number of program headers: 0
Size of section headers: 40 (bytes)
Number of section headers: 12
Section header string table index: 9
At this point, I was getting antsy and I decided to try and create an executable with the -mabi=apcs-gnu,
arm-softfloat-linux-uclibceabi-gcc -mabi=apcs-gnu -static -o /mnt/arm_uclibc/tmp/test /mnt/arm/tmp/test.c
And I get a linker error:
/usr/libexec/gcc/arm-softfloat-linux-uclibceabi/ld: error: Source object /tmp/ccDq2f6R.o has EABI version 0, but target /mnt/arm_uclibc/tmp/test has EABI version 5
/usr/libexec/gcc/arm-softfloat-linux-uclibceabi/ld: failed to merge target specific data of file /tmp/ccDq2f6R.o
collect2: ld returned 1 exit status
QUESTION: This leads me to believe that my EABI is incorrect and I need OABI. Is that right?
I believed that it was the case, so I began to look into uclibc through crossdev:
$ crossdev arm-softfloat-linux-uclibc -P -v
I am happy to report that the files that DO get compiled into some sort of object binary have the right elf header. So I think this is what I want.
But this dies during compilation for uclibc as follows:
make[1]: `lib/ld-uClibc.so' is up to date.
LD libuClibc-0.9.33.2.so
libc/libc_so.a(_fpmaxtostr.os): In function `_fpmaxtostr':
_fpmaxtostr.c:(.text+0xbc): undefined reference to `__nedf2'
_fpmaxtostr.c:(.text+0xe0): undefined reference to `__eqdf2'
_fpmaxtostr.c:(.text+0xfc): undefined reference to `__divdf3'
_fpmaxtostr.c:(.text+0x108): undefined reference to `__ltdf2'
_fpmaxtostr.c:(.text+0x17c): undefined reference to `__muldf3'
_fpmaxtostr.c:(.text+0x348): undefined reference to `__gedf2'
_fpmaxtostr.c:(.text+0x40c): undefined reference to `__fixunsdfsi'
libc/libc_so.a(__psfs_do_numeric.os): In function `__psfs_do_numeric':
__psfs_do_numeric.c:(.text+0x534): undefined reference to `__truncdfsf2'
libc/libc_so.a(close.oS):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'
collect2: ld returned 1 exit status
make: *** [lib/libc.so] Error 1
If I have broken the error down properly, I believe that
1) The arm Makefile.arch is not properly building __aeabi_unwind_cpp_pr0 because the file is only built when EABI is set:
$ find . -name 'Makefile.arch' -exec grep -i -H -n 'pr1' "{}" \;
./uclibc-0.9.33.2/work/uClibc-0.9.33.2/libc/sysdeps/linux/arm/Makefile.arch:45: $(ARCH_OUT)/aeabi_sighandlers.os $(ARCH_OUT)/aeabi_unwind_cpp_pr1.o
$ find . -name 'aeabi_unwind_cpp_pr1.c*'
./uclibc-0.9.33.2/work/uClibc-0.9.33.2/libc/sysdeps/linux/arm/aeabi_unwind_cpp_pr1.c
$ cat ./uclibc-0.9.33.2/work/uClibc-0.9.33.2/libc/sysdeps/linux/arm/aeabi_unwind_cpp_pr1.c
#include <stdlib.h>
attribute_hidden void __aeabi_unwind_cpp_pr0 (void);
attribute_hidden void __aeabi_unwind_cpp_pr0 (void)
{
}
attribute_hidden void __aeabi_unwind_cpp_pr1 (void);
attribute_hidden void __aeabi_unwind_cpp_pr1 (void)
{
}
attribute_hidden void __aeabi_unwind_cpp_pr2 (void);
attribute_hidden void __aeabi_unwind_cpp_pr2 (void)
{
}
I believe the fix for this error is:
--- Makefile.arch.old 2012-05-28 00:43:52.918708833 -0500
+++ Makefile.arch.new 2012-05-28 00:44:30.658708443 -0500
@@ -42,5 +42,6 @@
libc-static-y += $(ARCH_OUT)/aeabi_lcsts.o $(ARCH_OUT)/aeabi_math.o \
$(ARCH_OUT)/aeabi_sighandlers.o
libc-nonshared-y += $(ARCH_OUT)/aeabi_lcsts.os $(ARCH_OUT)/aeabi_math.os \
- $(ARCH_OUT)/aeabi_sighandlers.os $(ARCH_OUT)/aeabi_unwind_cpp_pr1.o
+ $(ARCH_OUT)/aeabi_sighandlers.os
endif
+libc-nonshared-y += $(ARCH_OUT)/aeabi_unwind_cpp_pr1.o
2) The soft-float in gcc is not properly being included either by the linker. I can't really tell why at this point.
$ find . -name '*.c' -exec grep -i -H -n nedf2 "{}" \;
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/mips/mips.c:11123: set_optab_libfunc (ne_optab, DFmode, "__mips16_nedf2");
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/eqdf2.c:51:strong_alias(__eqdf2, __nedf2);
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/testsuite/gcc.c-torture/execute/gofast.c:32:int nedf2 (double a, double b) { return a != b; }
./gcc-4.5.3-r2/work/gcc-4.5.3/libgcc/config/rx/rx-abi-functions.c:41:int _COM_CMPNEd (double a, double b) { return __nedf2 (a, b) != 0; }
$ ls ./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/
README double.h extendsftf2.c fixsfti.c fixunssfdi.c floatdisf.c floattitf.c floatuntidf.c lesf2.c negtf2.c single.h trunctfdf2.c
adddf3.c eqdf2.c extendxftf2.c fixtfdi.c fixunssfsi.c floatditf.c floatundidf.c floatuntisf.c letf2.c op-1.h soft-fp.h trunctfsf2.c
addsf3.c eqsf2.c fixdfdi.c fixtfsi.c fixunssfti.c floatsidf.c floatundisf.c floatuntitf.c muldf3.c op-2.h subdf3.c trunctfxf2.c
addtf3.c eqtf2.c fixdfsi.c fixtfti.c fixunstfdi.c floatsisf.c floatunditf.c gedf2.c mulsf3.c op-4.h subsf3.c unorddf2.c
divdf3.c extenddftf2.c fixdfti.c fixunsdfdi.c fixunstfsi.c floatsitf.c floatunsidf.c gesf2.c multf3.c op-8.h subtf3.c unordsf2.c
divsf3.c extended.h fixsfdi.c fixunsdfsi.c fixunstfti.c floattidf.c floatunsisf.c getf2.c negdf2.c op-common.h t-softfp unordtf2.c
divtf3.c extendsfdf2.c fixsfsi.c fixunsdfti.c floatdidf.c floattisf.c floatunsitf.c ledf2.c negsf2.c quad.h truncdfsf2.c
$ grep -i -H -n nedf2 ./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/*
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/eqdf2.c:51:strong_alias(__eqdf2, __nedf2);
$ grep -i -H -n eqdf2 ./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/*
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/eqdf2.c:35:CMPtype __eqdf2(DFtype a, DFtype b)
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/eqdf2.c:51:strong_alias(__eqdf2, __nedf2);
$ grep -i -H -n divdf3 ./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/*
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/divdf3.c:35:DFtype __divdf3(DFtype a, DFtype b)
$ grep -i -H -n ltdf2 ./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/*
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/ledf2.c:51:strong_alias(__ledf2, __ltdf2);
$ grep -i -H -n muldf3 ./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/*
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/muldf3.c:35:DFtype __muldf3(DFtype a, DFtype b)
$ grep -i -H -n gedf2 ./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/*
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/gedf2.c:35:CMPtype __gedf2(DFtype a, DFtype b)
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/gedf2.c:51:strong_alias(__gedf2, __gtdf2);
$ grep -i -H -n fixunsdfsi ./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/*
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/fixunsdfsi.c:35:USItype __fixunsdfsi(DFtype a)
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/t-softfp:71:softfp_func_list := $(filter-out floatdidf floatdisf fixunsdfsi fixunssfsi \
$ grep -i -H -n truncdfsf2 ./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/*
./gcc-4.5.3-r2/work/gcc-4.5.3/gcc/config/soft-fp/truncdfsf2.c:36:SFtype __truncdfsf2(DFtype a)
So I tried to force GCC to build for a soft-float and get that linked later within uclibc's build:
$ UCLIBC_CPU=ARM926T ACCEPT_KEYWORDS="arm" CPU_CFLAGS="-marm -march=armv5te -mtune=arm926ej-s -mabi=apcs-gnu -mno-thumb" EXTRA_FLAGS="-msoft-float -mfloat-abi=soft" UCLIBC_EXTRA_CFLAGS="${CPU_CFLAGS} ${EXTRA_CFLAGS}" STAGE1_CFLAGS="${EXTRA_CFLAGS}" CFLAGS="${EXTRA_CFLAGS}" crossdev -A arm -t arm-softfloat-linux-uclibc -P -v
And then I checked to see if -msoft-float and -mfloat-abi=soft were used within any log for compiling.
$ find . -name '*.log' -exec grep -i -H -n msoft-float "{}" \;
<nothing>
$ find . -name '*.log'
./work/build/arm-softfloat-linux-uclibc/libgcc/config.log
./work/build/libcpp/config.log
./work/build/gcc/config.log
./work/build/fixincludes/config.log
./work/build/intl/config.log
./work/build/build-x86_64-pc-linux-gnu/libiberty/config.log
./work/build/build-x86_64-pc-linux-gnu/fixincludes/config.log
./work/build/libdecnumber/config.log
./work/build/libiberty/config.log
./work/build/config.log
./work/gcc-4.5.3/contrib/reghunt/examples/29478.log
./work/gcc-4.5.3/contrib/reghunt/examples/29906a.log
./work/gcc-4.5.3/contrib/reghunt/examples/29906b.log
./work/gcc-4.5.3/contrib/reghunt/examples/28970.log
./work/gcc-4.5.3/contrib/reghunt/examples/29106.log
./work/gcc-4.5.3/contrib/reghunt/examples/30643.log
./temp/elibtool.log
./temp/epatch_user.log
./temp/epatch.log
./temp/eclass-debug.log
./temp/build.log
But I do note that --with-float=soft is set within the config.log, so that makes me believe that the float should have been generated.
And I note the -D__GCC_FLOAT_NOT_NEEDED in the compilation options for gcc.
I ran a regression on GCC to see where the break occurred.
- gcc 4.x does not work with uclibc. -- starting with 4.4.4-r2, uclibc has a linking failure with gcc -- prior to 4.4.4, gcc does not appear to
- gcc-3.4.6-r2 does work provided that USE=-nptl is used
For reference I ran: binutils: 2.22-r1 Linux Header: 3.3, 3.4 uclibc: 0.9.33.2 gcc: 3.2.3-r4, 3.3.6-r1, 3.4.6-r2, 4.1.4-r1, 4.3.3-r2, 4.4.2, 4.4.4-r2, 4.4.5, 4.4.6-r1, 4.4.7, 4.5.3-r2, 4.6.0, 4.6.1-r1, 4.6.2, 4.6.3
I ended up moving to gcc-4.7.0. The final build is:
My working crossdev arm-softfloat-linux-uclibc compiler command looks like this:
The options above might not all be needed, but as a note, the next major version of gcc is going to obsolete uclibc's OABI unless they end up with a developer that's willing to keep it updated. There hasn't been one in at least two years.
Also, I do not have nptl enabled with gcc. I think it is possible to do a crossdev build stage 3 without nptl enabled and then do the final gcc with nptl, but I have not done any testing with the binaries to prove that this actually inserts nptl.
Unfortunately, gcc and uclibc do not play nice together right out of the box. I also realized I needed some patches to fix the errors I found.
To make this easier on myself, I also ended up creating a patches folder so I didn't have to create an entirely new overlay with ebuilds. During the execution of emerge, this let me add patches on the fly. For the most part, this will work with most packages. I'm still working with Python to make that a smooth compile.
Creating the patches folder:
Updating portage bashrc:
Also need to create bashrc.autopatch.
Creating a patch for gcc:
Creating a patch for uclibc: