Add 64 bit offset to a pointer

.net, f#, interop, native, pointers

Solution

I'm not an expert in this field, but I took a look at the F# implementation of the `NativePtr` module and I think that there is no performance overhead associated with converting `nativeptr<'a>` to `nativeint` and back.

The implementation uses inline IL and the inline IL code doesn't contain any code - it is there just to make the F# compiler think that the value on the stack has a different type:

let inline ofNativeInt (x:nativeint)    = (# "" x : nativeptr<_> #)
let inline toNativeInt (x:nativeptr<_>) = (# "" x : nativeint    #)

In fact, the `NativePtr.add` method also uses these two methods - it converts the pointer to `nativeint` and then adds the 32bit integer (multiplied by the size of the `'a` type).

So, the following function should be fine:

let inline addNativeInt (x:nativeptr<'a>) (n:nativeint) : nativeptr<'a> = 
   (NativePtr.toNativeInt x) + n |> NativePtr.ofNativeInt

All functions used in the code should be inlined, so you'll end up with just a single instruction for addition (though, I have not verified that). You don't even have to worry about using the function multiple times in your code (you can work with `nativeptr<'a>` all the time and use this function for addition).

However, partitioning data may also be an option - as far as I know, the MSR team that was using F# for processing some large (>2GB) data sets used exactly this approach - they partitioned the data into 2GB blocks (stored in arrays).

Problem

In F#, there's the NativePtr module, but it seems to only support 32 bit offsets for its’ add/get/set functions, just like System.IntPtr does. Is there a way to add a 64 bit offset to a native pointer (nativeptr<'a>) in F#? Of course I could convert all addresses to 64 bit integers, do normal integer operations and then convert the result again to nativeptr<'a>, but this would cost additional add and imul instructions. I really want the AGUs to perform the address calculations. For instance, using unsafe in C# you could do something like ``` void* ptr = Marshal.AllocHGlobal(...).ToPointer(); int64 offset = ...; T* newAddr = (T*)ptr + offset; // T has to be an unmanaged type ``` Well actually you can't, because there is no "unmanaged" constraint for type parameters, but at least you can do general pointer arithmetic in a non-generic way. In F# we finally got the unmanaged constraint; but how do I do the pointer arithmetic?

Original source