Friday, March 12, 2010

Hugepages patch

Got something working:

How to run:

# echo 200 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
# mkdir /dev/huge
# mount -t hugetlbfs none /dev/huge -o size=800M,mode=777
$ sbcl --hugetlbfs-path /dev/huge --huge-pages-nr 200
Here 800 megabytes of memory is prepared for use as huge pages. SBCL has two new command line switches now. If there's something wrong with hugetlbfs, SBCL will fallback to "usual" memory.

To extend feature for other OSes, where POSIX API for hugepages is available, you need to add only few bits:

diff --git a/src/runtime/linux-os.c b/src/runtime/linux-os.c
index 9b2e3cf..386fc25 100644
--- a/src/runtime/linux-os.c
+++ b/src/runtime/linux-os.c
@@ -56,6 +56,8 @@
 #include "cheneygc-internal.h"
+#include "hugepages.h"
 #ifdef LISP_FEATURE_X86
 /* Prototype for personality(2). Done inline here since the header file
  * for this isn't available on old versions of glibc. */
@@ -313,7 +315,10 @@ os_validate(os_vm_address_t addr, os_vm_size_t len)
-    actual = mmap(addr, len, OS_VM_PROT_ALL, flags, -1, 0);
+    actual = huge_page_validate(addr, len);
+    if (!actual)
+        actual = mmap(addr, len, OS_VM_PROT_ALL, flags, -1, 0);
     if (actual == MAP_FAILED) {
         return 0;               /* caller should check this */
@@ -337,6 +342,7 @@ os_validate(os_vm_address_t addr, os_vm_size_t len)
 os_invalidate(os_vm_address_t addr, os_vm_size_t len)
+    huge_page_invalidate(addr, len);
     if (munmap(addr,len) == -1) {

In other words, only new header and small modification of os_validate and os_invalidate are required.

UPD: Garbage collector works much slower with the patch and needs further investigation.

SBCL and huge pages

Yesterday was a lucky day: I compiled SBCL with sb!vm:*backend-page-bytes* = 2097152. There were several problems preventing doing this, but, hopefully, all of them are solved. At least, I'm able to run new core and do everything, what comes in mind, for example, load ZeroMQ2 bindings.

Next step is to teach memory allocator to mmap() pages from hugetlbfs. That will give SBCL real 2mb pages, not 2mb regions composed of 4kb chunks.

Stay tuned!