Are there compiler flags to get malloc to return pointers above the 4G limit for 64bit testing (various platforms)?
64-bit, c++, malloc, testing
Solution
One technique I have used in the past is to allocate enough memory at startup that all the address space below the 4GB limit is used up. While this technique does rely on malloc first using the lower parts of the address space, this was true on all the platforms I work on (Linux, Solaris and Windows).
Because of how Linux uses overcommit, if you don't touch the allocated space below the 4GB limit, you won't use up any virtual memory.
On Windows, you can use VirtualAlloc() with the MEM_RESERVE flag to use up address space without allocating any actual storage.
Problem
I need to test code ported from 32bit to 64bit where pointers are cast around as integer handles, and I have to make sure that the correct sized types are used on 64 bit platforms. Are there any flags for various compilers, or even flags at runtime which will ensure that `malloc` returns pointer values greater than the 32bit limit? Platforms I'm interested in: - Visual Studio 2008 on Windows XP 64, and other 64 bit windows - AIX using xLC - 64bit gcc - 64bit HP/UX using aCC Sample Application that allocates 4GB So thanks to R Samuel Klatchko's answer, I was able to implement a simple test app that will attempt to allocate pages in the first 4GB of address space. Hopefully this is useful to others, and other SO users can give me an idea how portable/effective it is. ``` #include <stdlib.h> #include <stdio.h> #define UINT_32_MAX 0xFFFFFFFF #ifdef WIN32 typedef unsigned __int64 Tuint64; #include <windows.h> #else typedef unsigned long long Tuint64; #include <sys/mman.h> #include <unistd.h> #include <errno.h> #include <string.h> #endif static void* Allocate(void* pSuggested, unsigned int PageSize) { #ifdef WIN32 void* pAllocated = ::VirtualAlloc(pSuggested, PageSize, MEM_RESERVE ,PAGE_NOACCESS); if (pAllocated) { return pAllocated; } return (void*)-1; #else void* pAllocated = ::mmap(pSuggested, PageSize, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0); if (pAllocated == MAP_FAILED) { pAllocated = (void*)-1; } return pAllocated; #endif } static void Deallocate(void* pRegion, unsigned int PageSize) { #ifdef WIN32 ::VirtualFree(pRegion,0,MEM_RELEASE); #else ::munmap(pRegion,PageSize); #endif } static void Gobble32bitAddressSpace() { #ifdef WIN32 SYSTEM_INFO SysInfo; ::GetSystemInfo(&SysInfo); unsigned int PageSize = SysInfo.dwAllocationGranularity; #else unsigned int PageSize = ::sysconf(_SC_PAGE_SIZE); #endif unsigned int AllocatedPages = 0; unsigned int SkippedPages = 0; void *pStart = 0; while( ((Tuint64)pStart) < UINT_32_MAX) { void* pAllocated = Allocate(pStart, PageSize); if (pAllocated != (void*)-1) { if (pAllocated == pStart) { //Allocated at expected location AllocatedPages++; } else { //Allocated at a different location //unallocate and consider this page unreserved SkippedPages++; Deallocate(pAllocated,PageSize); } } else { //could not allocate at all SkippedPages++; } pStart = (char*)pStart + PageSize; } printf("PageSize : %u\n",PageSize); printf("Allocated Pages : %u (%u bytes)\n",AllocatedPages,PageSize*AllocatedPages); printf("Skipped Pages : %u (%u bytes)\n",SkippedPages,SkippedPages*PageSize); } int main() { Gobble32bitAddressSpace(); //Try to call malloc now and see if we get an //address above 4GB void* pFirstMalloc = ::malloc(1024); if (((Tuint64)pFirstMalloc) >= UINT_32_MAX) { printf("OK\n"); } else { printf("FAIL\n"); } return 0; } ```