Assembling 32-bit binaries on a 64-bit system (GNU toolchain)

assembly, att, build, linux, x86

Solution

TL:DR: use `gcc -m32 -static -nostdlib foo.S` (or equivalent as and ld options). Or if you don't define your own `_start`, just `gcc -m32 -no-pie foo.S`

You may need to install `gcc-multilib` if you link libc, or however your distro packages `/usr/lib32/libc.so`, `/usr/lib32/libstdc++.so` and so on. But if you define your own `_start` and don't link libraries, you don't need the library package, just a kernel that supports 32-bit processes and system calls. This includes most distros, but not Windows Subsystem for Linux v1.

Don't use `.code32`

`.code32` does not change the output file format, and that's what determines the mode your program will run in. It's up to you to not try to run 32bit code in 64bit mode. `.code32` is for assembling kernels that have some 16 and some 32-bit code, and stuff like that. If that's not what you're doing, avoid it so you'll get build-time errors when you build a `.S` in the wrong mode if it has any `push` or `pop` instructions, for example. `.code32` just lets you create confusing-to-debug runtime problems instead of build-time errors.

Suggestion: use the `.S` extension for hand-written assembler. (`gcc -c foo.S` will run it through the C preprocessor before `as`, so you can `#include <sys/syscall.h>` for syscall numbers, for example). Also, it distinguishes it from `.s` compiler output (from `gcc foo.c -O3 -S`).

To build 32-bit binaries, use one of these commands

gcc -g foo.S -o foo -m32 -nostdlib -static  # static binary with absolutely no libraries or startup code
                       # -nostdlib still dynamically links when Linux where PIE is the default, or on OS X

gcc -g foo.S -o foo -m32 -no-pie            # dynamic binary including the startup boilerplate code.
     # Use with code that defines a main(), not a _start

Documentation for `nostdlib`, `-nostartfiles`, and `-static`.

Using libc functions from `_start` (see the end of this answer for an example)

Some functions, like `malloc(3)`, or stdio functions including `printf(3)`, depend on some global data being initialized (e.g. `FILE *stdout` and the object it actually points to).

`gcc -nostartfiles` leaves out the CRT `_start` boilerplate code, but still links `libc` (dynamically, by default). On Linux, shared libraries can have initializer sections that are run by the dynamic linker when it loads them, before jumping to your `_start` entry point. So `gcc -nostartfiles hello.S` still lets you call `printf`. For a dynamic executable, the kernel runs `/lib/ld-linux.so.2` on it instead of running it directly (use `readelf -a` to see the "ELF interpreter" string in your binary). When your `_start` eventually runs, not all the registers will be zeroed, because the dynamic linker ran code in your process.

However, `gcc -nostartfiles -static hello.S` will link, but crash at runtime if you call `printf` or something without calling glibc's internal init functions. (see Michael Petch's comment).

Of course you can put any combination of `.c`, `.S`, and `.o` files on the same command line to link them all into one executable. If you have any C, don't forget `-Og -Wall -Wextra`: you don't want to be debugging your asm when the problem was something simple in the C that calls it that the compiler could have warned you about.

Use `-v` to have gcc show you the commands it runs to assemble and link. To do it "manually":

as foo.S -o foo.o -g --32 &&      # skips the preprocessor
ld -o foo foo.o  -m elf_i386

file foo
foo: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped

`gcc -nostdlib -m32` is easier to remember and type than the two different options for as and ld (`--32` and `-m elf_i386`). Also, it works on all platforms, including ones where executable format isn't ELF. (But Linux examples won't work on OS X, because the system call numbers are different, or on Windows because it doesn't even use the `int 0x80` ABI.)

NASM/YASM

gcc can't handle NASM syntax. (`-masm=intel` is more like MASM than NASM syntax, where you need `offset symbol` to get the address as an immediate). And of course the directives are different (e.g. `.globl` vs `global`).

You can build with `nasm` or `yasm`, then link the `.o` with `gcc` as above, or `ld` directly.

I use a wrapper script to avoid the repetitive typing of the same filename with three different extensions. (nasm and yasm default to `file.asm` -> `file.o`, unlike GNU as's default output of `a.out`). Use this with `-m32` to assemble and link 32bit ELF executables. Not all OSes use ELF, so this script is less portable than using `gcc -nostdlib -m32` to link would be..

#!/bin/bash
# usage: asm-link [-q] [-m32] foo.asm  [assembler options ...]
# Just use a Makefile for anything non-trivial.  This script is intentionally minimal and doesn't handle multiple source files
# Copyright 2020 Peter Cordes.  Public domain.  If it breaks, you get to keep both pieces

verbose=1                       # defaults
fmt=-felf64
#ldopt=-melf_i386
ldlib=()

linker=ld
#dld=/lib64/ld-linux-x86-64.so.2
while getopts 'Gdsphl:m:nvqzN' opt; do
    case "$opt" in
        m)  if [ "m$OPTARG" = "m32" ]; then
                fmt=-felf32
                ldopt=-melf_i386
                #dld=/lib/ld-linux.so.2  # FIXME: handle linker=gcc non-static executable
            fi
            if [ "m$OPTARG" = "mx32" ]; then
                fmt=-felfx32
                ldopt=-melf32_x86_64
            fi
            ;;
        #   -static
        l)  linker="gcc -no-pie -fno-plt -nostartfiles"; ldlib+=("-l$OPTARG");;
        p)  linker="gcc -pie -fno-plt -nostartfiles"; ldlib+=("-pie");;
        h)  ldlib+=("-Ttext=0x200800000");;   # symbol addresses outside the low 32.  data and bss go in range of text
                          # strace -e raw=write  will show the numeric address
        G)  nodebug=1;;      # .label: doesn't break up objdump output
        d)  disas=1;;
        s)  runsize=1;;
        n)  use_nasm=1 ;;
        q)  verbose=0 ;;
        v)  verbose=1 ;;
        z)  ldlib+=("-zexecstack") ;;
        N)  ldlib+=("-N") ;;   # --omagic = read+write text section
    esac
done
shift "$((OPTIND-1))"   # Shift off the options and optional --

src=$1
base=${src%.*}
shift

#if [[ ${#ldlib[@]} -gt 0 ]]; then
    #    ldlib+=("--dynamic-linker" "$dld")
    #ldlib=("-static" "${ldlib[@]}")
#fi

set -e
if (($use_nasm)); then
  #  (($nodebug)) || dbg="-g -Fdwarf"     # breaks objdump disassembly, and .labels are included anyway
    ( (($verbose)) && set -x    # print commands as they're run, like make
    nasm "$fmt" -Worphan-labels $dbg  "$src" "$@" &&
        $linker $ldopt -o "$base" "$base.o"  "${ldlib[@]}")
else
    (($nodebug)) || dbg="-gdwarf2"
    ( (($verbose)) && set -x    # print commands as they're run, like make
    yasm "$fmt" -Worphan-labels $dbg "$src" "$@" &&
        $linker $ldopt -o "$base" "$base.o"  "${ldlib[@]}" )
fi

# yasm -gdwarf2 includes even .local labels so they show up in objdump output
# nasm defaults to that behaviour of including even .local labels

# nasm defaults to STABS debugging format, but -g is not the default

if (($disas));then
    objdump -drwC -Mintel "$base"
fi

if (($runsize));then
    size $base
fi

I prefer YASM for a few reasons, including that it defaults to making long-`nop`s instead of padding with many single-byte `nop`s. That makes for messy disassembly output, as well as being slower if the nops ever run. (In NASM, you have to use the `smartalign` macro package.)

However, YASM hasn't been maintained for a while and only NASM has AVX512 support; these days I more often just use NASM.

Example: a program using libc functions from _start

# hello32.S

#include <asm/unistd_32.h>   // syscall numbers.  only #defines, no C declarations left after CPP to cause asm syntax errors

.text
#.global main   # uncomment these to let this code work as _start, or as main called by glibc _start
#main:
#.weak _start

.global _start
_start:
        mov     $__NR_gettimeofday, %eax  # make a syscall that we can see in strace output so we know when we get here
        int     $0x80

        push    %esp
        push    $print_fmt
        call   printf

        #xor    %ebx,%ebx                 # _exit(0)
        #mov    $__NR_exit_group, %eax    # same as glibc's _exit(2) wrapper
        #int    $0x80                     # won't flush the stdio buffer

        movl    $0, (%esp)   # reuse the stack slots we set up for printf, instead of popping
        call    exit         # exit(3) does an fflush and other cleanup

        #add    $8, %esp     # pop the space reserved by the two pushes
        #ret                 # only works in main, not _start

.section .rodata
print_fmt: .asciz "Hello, World!\n%%esp at startup = %#lx\n"
$ gcc -m32 -nostdlib hello32.S
/tmp/ccHNGx24.o: In function `_start':
(.text+0x7): undefined reference to `printf'
...
$ gcc -m32 hello32.S
/tmp/ccQ4SOR8.o: In function `_start':
(.text+0x0): multiple definition of `_start'
...

Fails at run-time, because nothing calls the glibc init functions. (`__libc_init_first`, `__dl_tls_setup`, and `__libc_csu_init` in that order, according to Michael Petch's comment. Other `libc` implementations exist, including MUSL which is designed for static linking and works without initialization calls.)

$ gcc -m32 -nostartfiles -static hello32.S     # fails at run-time
$ file a.out
a.out: ELF 32-bit LSB executable, Intel 80386, version 1 (GNU/Linux), statically linked, BuildID[sha1]=ef4b74b1c29618d89ad60dbc6f9517d7cdec3236, not stripped
$ strace -s128 ./a.out
execve("./a.out", ["./a.out"], [/* 70 vars */]) = 0
[ Process PID=29681 runs in 32 bit mode. ]
gettimeofday(NULL, NULL)                = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SI_KERNEL, si_addr=0} ---
+++ killed by SIGSEGV (core dumped) +++
Segmentation fault (core dumped)

You could also `gdb ./a.out`, and run `b _start`, `layout reg`, `run`, and see what happens.

$ gcc -m32 -nostartfiles hello32.S             # Correct command line
$ file a.out
a.out: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, BuildID[sha1]=7b0a731f9b24a77bee41c13ec562ba2a459d91c7, not stripped

$ ./a.out
Hello, World!
%esp at startup = 0xffdf7460

$ ltrace -s128 ./a.out > /dev/null
printf("Hello, World!\n%%esp at startup = %#lx\n", 0xff937510)      = 43    # note the different address: Address-space layout randomization at work
exit(0 <no return ...>
+++ exited (status 0) +++

$ strace -s128 ./a.out > /dev/null        # redirect stdout so we don't see a mix of normal output and trace output
execve("./a.out", ["./a.out"], [/* 70 vars */]) = 0
[ Process PID=29729 runs in 32 bit mode. ]
brk(0)                                  = 0x834e000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
....   more syscalls from dynamic linker code
open("/lib/i386-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
mmap2(NULL, 1814236, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xfffffffff7556000    # map the executable text section of the library
... more stuff
# end of dynamic linker's code, finally jumps to our _start

gettimeofday({1461874556, 431117}, NULL) = 0
fstat64(1, {st_mode=S_IFCHR|0666, st_rdev=makedev(1, 3), ...}) = 0  # stdio is figuring out whether stdout is a terminal or not
ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0xff938870) = -1 ENOTTY (Inappropriate ioctl for device)
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xfffffffff7743000      # 4k buffer for stdout
write(1, "Hello, World!\n%esp at startup = 0xff938fb0\n", 43) = 43
exit_group(0)                           = ?
+++ exited with 0 +++

If we'd used `_exit(0)`, or made the `sys_exit` system call ourselves with `int 0x80`, the `write(2)` wouldn't have happened. With stdout redirected to a non-tty, it defaults to full-buffered (not line-buffered), so the `write(2)` is only triggered by the `fflush(3)` as part of `exit(3)`. Without redirection, calling `printf(3)` with a string containing newlines will flush immediately.

Behaving differently depending on whether stdout is a terminal can be desirable, but only if you do it on purpose, not by mistake.

Problem

I wrote assembly code that successfully compiles: ``` as power.s -o power.o ``` However, it fails when I try to link the object file: ``` ld power.o -o power ``` In order to run on the 64 bit OS (Ubuntu 14.04), I added `.code32` at the beginning of the `power.s` file, however I still get the error: Segmentation fault (core dumped) `power.s`: ``` .code32 .section .data .section .text .global _start _start: pushl $3 pushl $2 call power addl $8, %esp pushl %eax pushl $2 pushl $5 call power addl $8, %esp popl %ebx addl %eax, %ebx movl $1, %eax int $0x80 .type power, @function power: pushl %ebp movl %esp, %ebp subl $4, %esp movl 8(%ebp), %ebx movl 12(%ebp), %ecx movl %ebx, -4(%ebp) power_loop_start: cmpl $1, %ecx je end_power movl -4(%ebp), %eax imull %ebx, %eax movl %eax, -4(%ebp) decl %ecx jmp power_loop_start end_power: movl -4(%ebp), %eax movl %ebp, %esp popl %ebp ret ```

Original source

Related problems