How does kernel get an executable binary file running under linux?

Question

How does kernel get an executable binary file running under linux? It seems a simple question, but anyone can help me dig deep? How the file is loaded to memory and how execution code get started? Can anyone help me and tell what's happening step by step?

Accepted Answer

Best moments of the `exec` system call on Linux 4.0

The best way to find all of that out is to GDB step debug the kernel with QEMU: How to debug the Linux kernel with GDB and QEMU?

`fs/exec.c` defines the system call at `SYSCALL_DEFINE3(execve`

Simply forwards to `do_execve`.

`do_execve`

Forwards to `do_execveat_common`.

`do_execveat_common`

To find the next major function, track when return value `retval` is last modified.

Starts building a `struct linux_binprm *bprm` to describe the program, and passes it to `exec_binprm` to execute.

`exec_binprm`

Once again, follow the return value to find the next major call.

`search_binary_handler`

Handlers are determined by the first magic bytes of the executable.

The two most common handlers are those for interpreted files (`#!` magic) and for ELF (`\x7fELF` magic), but there are other built-into the kernel, e.g. `a.out`. And users can also register their own though /proc/sys/fs/binfmt_misc

The ELF handler is defined at `fs/binfmt_elf.c`.

See also: Why do people write the #!/usr/bin/env python shebang on the first line of a Python script?

The `formats` list contains all the handlers.

Each handler file contains something like:

```
static int __init init_elf_binfmt(void)
{
    register_binfmt(&elf_format);
    return 0;
}
```

and `elf_format` is a `struct linux_binfmt` defined in that file.

`__init` is magic and puts that code into a magic section that gets called when the kernel starts: What does __init mean in the Linux kernel code?

Linker-level dependency injection!

There is also a recursion counter, in case an interpreter executes itself infinitely.

Try this:

```
echo '#!/tmp/a' > /tmp/a
chmod +x /tmp/a
/tmp/a
```

Once again we follow the return value to see what comes next, and see that it comes from:

```
retval = fmt->load_binary(bprm);
```

where `load_binary` is defined for each handler on the struct: C-style polymorsphism.

`fs/binfmt_elf.c:load_binary`

Does the actual work:

- parse the ELF file according to the ELF specification, here is an overview of the ELF file format: How to make an executable ELF file in Linux using a hex editor?

- set up the process initial program state based on the parsed ELF file, most notably:

- initial register setup in a `struct pt_regs`

- initial virtual memory setup, the memory is specified in the ELF segments: What's the difference of section and segment in ELF file format

- call `start_thread`, which marks the process as available to get to be scheduled by the scheduler

eventually the scheduler decides to run the process, and it must then jump to the PC address stored in `struct pt_regs` while also moving to a less privileged CPU state such as Ring 3 / EL0: What are Ring 0 and Ring 3 in the context of operating systems?

The scheduler gets woken up periodically by a clock hardware that generates interrupts periodically as configured earlier by the kernel, for example the old x86 PIT or the ARM timer. The kernel also registers handlers which run the scheduler code when the timer interrupts are fired.

TODO: continue source analysis further. What I expect to happen next:

- the kernel parses the INTERP header of the ELF to find the dynamic loader (usually set to `/lib64/ld-linux-x86-64.so.2`).

- if it is present:

- the kernel mmaps the dynamic loader and the ELF to be executed to memory

- dynamic loader is started, taking a pointer to the ELF in memory.

- now in userland, the loader somehow parses elf headers, and does `dlopen` on them

- `dlopen` uses a configurable search path to find those libraries (`ldd` and friends), mmap them to memory, and somehow inform the ELF where to find its missing symbols

- loader calls the `_start` of the ELF

otherwise, the kernel loads the executable into memory directly without the dynamic loader.

It must therefore in particular check if the executable is PIE or not an if it is place it in memory at a random location: What is the -fPIE option for position-independent executables in gcc and ld?

How does kernel get an executable binary file running under linux?

Solution

Problem

Related problems