Few words about Lisp

Saturday, January 12, 2013

Spartan SP605 jtag cable and Linux

Had to spend some time googling for answer of how to convince Xilinx Impact to see jtag cable in SP605 devkit. I didn't run driver setup script, instead figured out what Impact wants exactly with help of strace.

So far, you need two hex files to be in one of two places where Impact expects them, plus two udev rules to load firmware into adapter. You need fxload to do that, btw.

So, copy xusb_emb.hex and xusbdfwu.hex from Xilinx/[your version]/ISE_DS/ISE/data to /usr/share. This is very stupid, because strace shows that Impact has no problem finding these files in setup directory. Next, make sure there's a symlink from your actual libusb-1.0.so.x.y.z to libusb.so. Yeah, that's also a bit stupid step. On SL-6.3 I did:

# ln -s /usr/lib64/libusb-1.0.so.0.1.0 /usr/lib64/libusb.so

Create /etc/udev/rules.d/xusbdfwu.rules:

ATTRS{idVendor}=="03fd", ATTRS{idProduct}=="0008", MODE="666"
SUBSYSTEMS=="usb", ENV{DEVTYPE}=="usb_device", ACTION=="add", ATTRS{idVendor}=="03fd", ATTRS{idProduct}=="000d", RUN+="/sbin/fxload -v -t fx2 -I /usr/share/xusb_emb.hex -D $tempnode"

Though udev needs only xusb_emb.hex, Impact loads the other hex file as well.

In case you have troubles of getting device 03fd:000d (lsusb) to become a 03fd:0008, you can kill udevd and run it in console with --debug. You'll see what udev does not like about your rule.

Back to my gates.

Wednesday, September 26, 2012

Quartus jtagd and Linux kernel 3.5

Usbfs support, which is used by jtagd in Quartus, was dropped completely in kernel 3.5. I've seen quite invasive suggestion to patch path in jtagd's binary in one Altera forum, but there's a really simple workaround:

# mount -t debugfs none /sys/kernel/debug
# ln -s /sys/kernel/debug/usb/devices /dev/bus/usb
# mount /dev/bus /proc/bus -o bind

I've lost /proc/bus/input and /proc/bus/pci, of course, but I'm not sure if something in the system uses it. Anyway, jtag works again.

$ uname -r
$ jtagconfig
1) USB-Blaster [USB 1-]
  024030DD   EP4SGX530/ES

Friday, August 24, 2012

MSP430 USB Example on Linux and msp430-gcc

I had recently a small side project working with MSP430 microcontroller. For this project I did a port of one of TI's USB examples to standard GCC toolchain. It was not an easy task so I want to share my knowledge.

I have 64 bit Archlinux distro on my laptop, MSP430F5522-based custom board and TI's MSP-FET430UIF interface. Port was done for/MSP430_USB_API_Stacks/examples/cdcExamples/CCS/C1_Example from  MSP430USBDevleopersPackage_3_20_00.zip.

First I had to install next libraries (from AUR):

binutils-msp430 2.21.1a-2
gcc-msp430 4.6.3-1
msp430-gdb 7.2a-3
msp430-libc 20120224-2
msp430mcu 20120406-2

For debugging purposes there's some proprietary gdb proxy, but luckily I found opensource replacement called mspdebug. Archlinux has one, unfortunately it builds as a 64 bit binary, and that does not work with FET I used, because it has V3 firmware (version 30204005), which required mspdebug talking to FET via tilib protocol provided by proprietary libmsp430.so library. Likely, TI gives sources, and anybody can build it, if dare.

I got sources: http://www.ti.com/lit/sw/slac460c/slac460c.zip plus some Linux-related bits: http://superb-sea2.dl.sourceforge.net/project/msp4linux/msp430_add_alpha_0.6.1.tar.bz2

This software must be built in 32 bit mode and linked against 32 bit libraries. It relies on C++ Boost, so I had to install both lib32-boost-libs (runtime libraries) and boost-1.50 (headers). Library uses singleton implementation taken from boost internal's, which where changed in that respect in 1.48. To overcome this problem I had to get boost-1.47 tarball, extract pool/detail/singleton.hpp and put it in /usr/include/boost/pool/detail/. Next I followed patch instructions from here: http://sourceforge.net/p/msp4linux/home/Home/, but before typing final `make -f Makefile.dyn' I had to patch makefile (don't copy-paste, it won't work, but this and next patches are very trivial):

--- Makefile.dyn   2012-07-18 11:33:59.000000000 -0400
+++ Makefile.dyn        2012-08-23 10:02:13.502627239 -0400
@@ -1,14 +1,14 @@
 CXX:= g++
 ifdef DEBUG
-CXXFLAGS += -g -O0
+CXXFLAGS += -g -O0 -m32
+CXXFLAGS += -Os -m32

During build it turned out that I to replace my gcc by gcc-multilib, otherwise boost screwed up.

Next step is to build 32 bit version of mspdebug. It is straightforward, just small patch was required:

diff --git a/Makefile b/Makefile
index 7ce7785..5bbf143 100644
--- a/Makefile
+++ b/Makefile
@@ -71,10 +71,10 @@ else
 INCLUDES = -I. -Isimio -Iformats -Idrivers -Iutil -Iui
-GCC_CFLAGS = -O1 -Wall -Wno-char-subscripts -ggdb
+GCC_CFLAGS = -O1 -Wall -Wno-char-subscripts -ggdb -m32

Also I had to install 32-bit versions of libusb, libreadline and libtinfo.so. I took these libraries from Fedora-17 repo, from these rpms: libusb-0.1.3-10.fc17.i686.rpm, readline-6.2-4.fc17.i686.rpm, ncurses-libs-5.9-4.20120204.fc17.i686.rpm. Get shared libraries out of packages, put in /usr/local/lib, make symlinks libusb.so -> libusb-0.1.so.4.4.4, libreadline.so -> libreadline.so.6.2, libtinfo.so -> libtinfo.so.5.9 and run ldconfig.

When you get mspdebug compiled, try to run it and see if it works. In my case I've seen:

$ mspdebug32 -d /dev/ttyACM0 tilib
MSPDebug version 0.20 - debugging tool for MSP430 MCUs
Copyright (C) 2009-2012 Daniel Beer
This is free software; see the source for copying conditions.  There is NO

MSP430_Initialize: /dev/ttyACM0
Firmware version is 30204005
MSP430_VCC: 3000 mV
VCC  in[mV]: 3000
VCC out[mV]: 3303
VCC out[mV]: 3314
num of devices 1
JtagID: 91
version ID (0x91): 2255
Device: MSP430F5522 (id = 0x0013)
8 breakpoints available
Chip ID data: 55 22 14

Available commands:
    =           erase       isearch     prog        setbreak    sym        
    alias       exit        load        read        setwatch    verify     
    break       fill        locka       regs        setwatch_r 
    cgraph      gdb         md          reset       setwatch_w 
    delbreak    help        mw          run         simio      
    dis         hexout      opt         set         step       

Available options:
    color           gdb_loop        iradix         
    fet_block_size  gdbc_xfer_size  quiet          

Type "help " for more information.
Press Ctrl+D to quit.


Grab demo sources at https://docs.google.com/open?id=0B73s0UHdKV4hWmtrUXZqWnB4U0k. In order to build it for MCU different than 5522, you have to specify correct version in autogen.sh and src/Makefile.am. If your host's gcc target is not x86_64-unknown-linux-gnu (look in gcc -v) then fix it in same autogen.sh file too. I failed to figure out how to use TI's linker script with GNU ld, so I've copied some constants as LDFLAGS by hands (look in msp430USB.cmd, adjust src/Makefile.am as appropriate). Run autogen.sh and make.

When demo is built successfully, run mspdebug in one terminal (might require root permissions or correct access permissions to device file):

$ mspdebug32 -d /dev/ttyACM0 tilib

In prompt type `gdb' or `gdb_loop' and hit Enter.

Copy these bits into ~/.gdbinit:

set remoteaddresssize 64
set remotetimeout 999999
set remote memory-write-packet-size 1024
set remote memory-write-packet-size fixed
set remote memory-read-packet-size 1024
target remote localhost:2000

Then run gdb:

$ msp430-gdb demo

Now you can flash your software into MCU (`load demo'), run it, debug it, etc.

Have fun!

Sunday, December 5, 2010

Status update

Not much news to the moment. Except I joined Lisp startup (NovaSparks) and moved to Boston, MA in the middle of November.

Friday, June 18, 2010

Why zero-copy is missing in CL-ZMQ

ZeroMQ can utilize zero-copy technique, which may save a lot of cpu ticks when using large messages. Unfortunately, it's not possible to get this feature to work with supported Common Lisp implementations due to a couple of reasons:
  • ZeroMQ uses threads for asynchronous I/O operations on messages. When it finishes all the work with message's data, it calls back Lisp to let it know that data can be safely ripped out.
Unfortunately, not all Lisps can distinguish and handle callback from non-lisp thread. For example, Clozure CL feels good here, but SBCL crashes and LispWorks just hangs.
  • Garbage collector may move lisp objects, which are currently used in I/O threads. This leads to image corruption or segmentation fault.
Widely used with-pinned-object technique is not working here, because ZeroMQ operates asynchronously and pinned body can't protect data from GC when it needs to. It's possible to turn off garbage collecting completely or design system in such way, that it won't do gc'ing when it shouldn't. But that's a hard way.

Other possibility is to allocate objects in static area, where garbage collector doesn't move live objects. Some Lisps support this feature, for example AllegroCL, LispWorks and GCL.

If both features (callback from non-lisp thread, objects in static area) are supported by certain Lisp implementation, it's possible then to use zero-copy in CL-ZMQ. Unfortunately, all four Lisps that I use for development and testing (SBCL, Clozure, CLISP and LispWorks) miss this combination.

Sunday, June 13, 2010

CL-ZMQ and LispWorks 6

Since yesterday I'm happy user of commercial version of LW.

LW complains about unshadowed symbol `identity'. Yeah, that's strange it wasn't yet catched by 3 other lisps I tested with (sbcl, clisp, ccl). Other minor problem was with `trivial-garbage' library: it uses `hcl:mark-and-sweep', but this function exists only on 32-bit LWs. 64-bit uses `system:marking-gc'. Also `system::hash-table-weak-kind' has a bit different name: `system::hash-table-%weak-kind'.

Otherwise everything works good. LW is officially supported by CL-ZMQ now :)

Friday, March 12, 2010

Hugepages patch

Got something working:


How to run:

# echo 200 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
# mkdir /dev/huge
# mount -t hugetlbfs none /dev/huge -o size=800M,mode=777
$ sbcl --hugetlbfs-path /dev/huge --huge-pages-nr 200
Here 800 megabytes of memory is prepared for use as huge pages. SBCL has two new command line switches now. If there's something wrong with hugetlbfs, SBCL will fallback to "usual" memory.

To extend feature for other OSes, where POSIX API for hugepages is available, you need to add only few bits:

diff --git a/src/runtime/linux-os.c b/src/runtime/linux-os.c
index 9b2e3cf..386fc25 100644
--- a/src/runtime/linux-os.c
+++ b/src/runtime/linux-os.c
@@ -56,6 +56,8 @@
 #include "cheneygc-internal.h"
+#include "hugepages.h"
 #ifdef LISP_FEATURE_X86
 /* Prototype for personality(2). Done inline here since the header file
  * for this isn't available on old versions of glibc. */
@@ -313,7 +315,10 @@ os_validate(os_vm_address_t addr, os_vm_size_t len)
-    actual = mmap(addr, len, OS_VM_PROT_ALL, flags, -1, 0);
+    actual = huge_page_validate(addr, len);
+    if (!actual)
+        actual = mmap(addr, len, OS_VM_PROT_ALL, flags, -1, 0);
     if (actual == MAP_FAILED) {
         return 0;               /* caller should check this */
@@ -337,6 +342,7 @@ os_validate(os_vm_address_t addr, os_vm_size_t len)
 os_invalidate(os_vm_address_t addr, os_vm_size_t len)
+    huge_page_invalidate(addr, len);
     if (munmap(addr,len) == -1) {

In other words, only new header and small modification of os_validate and os_invalidate are required.

UPD: Garbage collector works much slower with the patch and needs further investigation.