Friday, March 12, 2010

Hugepages patch

Got something working:

How to run:

# echo 200 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
# mkdir /dev/huge
# mount -t hugetlbfs none /dev/huge -o size=800M,mode=777
$ sbcl --hugetlbfs-path /dev/huge --huge-pages-nr 200
Here 800 megabytes of memory is prepared for use as huge pages. SBCL has two new command line switches now. If there's something wrong with hugetlbfs, SBCL will fallback to "usual" memory.

To extend feature for other OSes, where POSIX API for hugepages is available, you need to add only few bits:

diff --git a/src/runtime/linux-os.c b/src/runtime/linux-os.c
index 9b2e3cf..386fc25 100644
--- a/src/runtime/linux-os.c
+++ b/src/runtime/linux-os.c
@@ -56,6 +56,8 @@
 #include "cheneygc-internal.h"
+#include "hugepages.h"
 #ifdef LISP_FEATURE_X86
 /* Prototype for personality(2). Done inline here since the header file
  * for this isn't available on old versions of glibc. */
@@ -313,7 +315,10 @@ os_validate(os_vm_address_t addr, os_vm_size_t len)
-    actual = mmap(addr, len, OS_VM_PROT_ALL, flags, -1, 0);
+    actual = huge_page_validate(addr, len);
+    if (!actual)
+        actual = mmap(addr, len, OS_VM_PROT_ALL, flags, -1, 0);
     if (actual == MAP_FAILED) {
         return 0;               /* caller should check this */
@@ -337,6 +342,7 @@ os_validate(os_vm_address_t addr, os_vm_size_t len)
 os_invalidate(os_vm_address_t addr, os_vm_size_t len)
+    huge_page_invalidate(addr, len);
     if (munmap(addr,len) == -1) {

In other words, only new header and small modification of os_validate and os_invalidate are required.

UPD: Garbage collector works much slower with the patch and needs further investigation.


Xach said...

It doesn't save the runtime options to the core properly, does it?

13-49 said...

Not yet. However, it is still able to load old core (saved from hugepaged image) to normal pages.

Also the patch is far away from optimal. This is only proof-of-concept.