implementing memset in assembly x86

Question

I'm trying to implement memset in assembly x86. both with a copy by byte and by word, so I'd get two functions: `kmemset` and `kmemsetw` that I expose to my C code this way: ``` extern uint8_t* kmemset(uint8_t* dest, uint8_t val, uint8_t size); extern uint16_t* kmemsetw(uint16_t* dest, uint16_t val, uint8_t size); ``` problem is when I test it, I get a segmentation fault. I tried to debug with gdb but it doesn't seem to be able to step into the asm code. I would be happy if anyone could comment on the code. (`kmemset` is pretty similar so I didn't include it). ``` .intel_syntax noprefix .section .text .align 16 .global kmemsetw .type kmemsetw, @function kmemsetw: push ebp mov ebp, esp push edi push ecx push ebx xor eax, eax mov ebx, [ebp+4] mov ax, [ebp+8] mov ecx, [ebp+12] mov edi, ebx rep stosw mov eax, edi pop ebx pop ecx pop edi pop ebp ret ```

Accepted Answer

Your not using `ebx` in your procedure, why save it? It does not need to be saved. `ecx` is a volatile register, you do not need to save it.

As gnometorule, mentioned, your parameters in your proc are off.

Another biggie is that you do not restore the stack pointer at the end. Sure you `pop ebp`, but where do you `mov esp, ebp`??? You `mov ebp, esp` at the beginning, but never reverse it.

If you look at `memset`, it returns the pointer passed to the proc. So, this is wrong: `mov eax, edi` it should be: `mov eax, [ebp + 8]`. `rep stos?` increments the pointer in `edi` so if you return `edi`, the returned pointer is wrong.

But why even set up a stack frame for this tiny proc? Just use `esp`, since we need to save `edi` to the stack, the params in `esp` will be offset the same as if we set up a stack frame:

```
kmemset:      
    push    edi             ; proc uses edi, so save it.

mov     ecx, [esp + 16] ; size_t num
    mov     al, [esp + 12]  ; int value 
    mov     edi, [esp + 8]  ; void * ptr
    rep     stosb

mov     eax, [esp + 8]  ; return pointer
    pop     edi             ; restore edi
    ret                     ; let caller adjust stack
```

using `stosw` will be a bit different.

```
SomeProc:
    push    ebp
    mov     ebp, esp
    push    edi

; params are at:
     ;~ ebp + 8
     ;~ ebp + 12
     ;~ ebp + 16
     etc...
    ; ...
    ; ...
    ; ...

pop     edi
    ; the following 2 lines
    ; can be replaced with
    ; leave
    mov     esp, ebp
    pop     ebp
    ret
```

implementing memset in assembly x86

Solution

Problem