Is it possible to replace Loader of an OS? Any way to obtain the control over Loader?

linker, linux, loader, operating-system, windows

Solution

Since each of the answers & comments is giving useful information. I just compiled, all the answers & comments into a single post.

I was just wondering if it is possible to replace Loader (executable program loader not the boot loader) of an Operating System (Windows is my choice).

No, in windows process creation and the user-mode loader in ntdll are tied together (PsCreateProcess will directly map in ntdll and jump to it so that it can finish resolving modules and setting up the process), you cannot replace it.

but there are resources availbable describing the format and loading of processes.

Here is a quite old but still uptodate MSDN article regarding PE files ( exe + dll )

- Part I. An In-Depth Look into the Win32 Portable Executable File Format by Matt Pietrek (MSDN Magazine, February 2002)

- Part II. An In-Depth Look into the Win32 Portable Executable File Format by Matt Pietrek (MSDN Magazine, March 2002)

You can use this information to write an app that starts a given executable.

If you are more interested in linux and the elf format you will find all you need in google.

Is there any way through which I can obtain the control over the OS Loader? I mean, I want things it is doing to be visible to me(each and every step).

On Windows, you can get some visibility into the loader at work by enabling Loader Snaps. You do this with `gflags.exe` (part of Debugging Tools for Windows). There's a nice `gflags.exe` reference http://www.osronline.com/DDKx/ddtools/gflags_4n77.htm . With Show Loader Snaps enabled, you can see loader trace messages by starting the application under a debugger (WinDBG).

If you want to play with this sort of thing then Linux is the best way to go.

The loader is part of the kernal -- but as you have access to all the kernal source you can play with it to your hearts content.

The loaders for various binary formats are in `fs/binfmt_*.c` in the Linux source (`fs/binfmt_elf.c` is the loader used for executables in ELF format - ie. the vast majority).

The dynamic loader `/lib{,64}/ld-linux.so.2` is also used for dynamically linked binaries - it's an example of an "interpreter" as referenced by the code in binfmt_elf.c.

Linux has pluggable executable file formats, so it is possible to add an extra program loader which will do its own custom stuff with executable files, rather than the standard ones (ELF, shell scripts, binfmt_misc).

The `binfmt_misc` module allows you to write custom loaders for executable programs entirely in userspace; this is commonly used to execute non-native binaries or interpreted binaries such as Java, CLR executables etc.

On the other hand if you wanted to replace the ELF loader with something else you can make a binfmt module directly in the kernel. Look at `fs/binfmt_*` for examples. The ELF loader itself is in there.

Problem

Original source