Implementing ;CODE in AArch32 Jonesforth

So I got a new Raspberry Pi and me being me got sidetracked playing with a toy Forth implementation, pijFORTHos (https://github.com/organix/pijFORTHos),which is a standalone AArch32 port of Jonesforth (https://rwmj.wordpress.com/2010/08/07/jonesforth-git-repository/), which is/was an IA32-only affair. Of course I've always been amused by the idea of writing a kernel in Forth...so why not? Sadly, I probably won't do much with this...

Anyway. To cut to the chase, pijFORTHos was missing the ;CODE functionality from Jonesforth 47, which let you define native machine words in Forth... i.e. an assembler, basically. A couple of completely empty and useless examples that do nothing (and yet not crash) would look like:

: FOO ;CODE
: BAR $NEXT ;CODE

The later is redundant, since $NEXT is already emitted by ;CODE. The implementation is straighforward.. Although I took the liberty of sticking it into jonesforth.s instead of the Forth prelude, and in the actual commit I'm a bit smarter about defining $NEXT and the actual _NEXT/NEXT bits used by the Forth core itself. You wonder why bother emitting the NEXT bits inline instead of branching, but the later would take up 3 cells as well (ldr, bx and immed for ldr) and also involve a branch. Look at how the $NEXT word is defined. Isn't this crazy? It's an IMMEDIATE word that writes literals, which just happen to be machine code, at HERE, effectively compiling them into the current word definition when used in compiler mode (such as a colon definition).

@
@ $NEXT ( -- ) emits the _NEXT body at HERE, to be used
@ in ;CODE or ;CODE-defined words.
@
defword "$NEXT",F_IMM,ASMNEXT
       .int LIT
        ldr r0, [FIP], #4
       .int COMMA
       .int LIT
        ldr r1, [r0]
       .int COMMA
       .int LIT
        bx r1
       .int COMMA
       .int EXIT
@
@ Finishes a machine code colon definition in Forth, as
@ a really basic assembler.
@
defword ";CODE",F_IMM,SEMICODE
       .int ASMNEXT                      @ end the word with NEXT macro
       .int LATEST, FETCH, DUP           @ LATEST points to the compiled word
       .int HIDDEN                       @ unhide the compiled word
       .int DUP, TDFA, SWAP, TCFA, STORE @ set codeword to data instead of DOCOL
       .int LBRAC                        @ just like ";" exit compile mode
       .int EXIT

DABT/PABT handling.

Figured out why at least kernel panic handling of DABT/PABT cases was yielding garbage. Going to rewrite the handlers to use the existing SAVE_CONTEXT and RESTORE_CONTEXT. Jury still out on whether the VM trap handling really has to happen the way it is currently, but the code can surely be simplified.

In an unrelated project, investigating lost event upcalls in an x64 PV OS. It's very aggravating that the PV interface is not documented in a fashion that doesn't necessitate combing through sources to find out the call stack layout and the contract between the hypervisor and PV OS. For x86-32 at least there was a book. I shouldn't need to spend a week studying the x64 hypervisor to figure out how to write a PV OS for it.

Here's hoping I do a better job with xen3-arm-tegra.

A

Not such a long way.

Framebuffer support is in, which means that finally, I can almost ditch the serial cable for a lot of work. It also means you don't need a special Xoom (or potentially - some other Tegra2 device) with a serial port. Retail Xoom owners can now proudly do...ummm...something. Not a whole lot, yet ;-(.

Found a page fault abort within __divdi3, but couldn't really debug it since the DABT handler (or should I say - the CPU context save code) is woefully broken. So before I can figure out what's wrong with __divdi3, I need to fix the trap handlers.

But wait, Andrei, how could you have the timer working then??? The IRQ path is completely different (as is the CPU context structure manipulated). Why? No clue... Won't be when I'm done with it ;-).

Btw, ARMv8 will support a division instruction. Until then, I'll have an excuse to read Knuth.

https://plus.google.com/114030412547992895028/posts/Y1MsnBwKCMX

A

On my long way to a framebuffer console...

Here I was hoping to have the framebuffer console done so I can finally move to working on more interesting aspects of my ARMv7 Xen3 rewrite, but alas before getting there I had to make the console layer more generic.

For an extra bonus I've simplified and cleaned up the serial layer. The whole L/H muxing to handle gdb and dom0/xen console traffic was a little insane :-). I'd rather use "O" gdb packets out (btw is this extension officially documented anywhere?)

Can't wait to move on...but having visible things to show people is cool too ;-).

I'm back

So I've decided I should continue to write here. After all, I work on relatively interesting, exciting to me things I feel I need to share with the world :-).

Anyway.

It's been an interesting summer. After attending the Linux Symposium in Ottawa with three papers [0] and being again a Tiano Core mentor for Google Summer of Code, I'm back to hacking things filesystem and virtualization related.

My Xen port to Tegra2 has been seeing a lot of attention from me. It's hard to call it a port, as I'm mostly approaching it from rewrite-everything angle, especially given the research prototype status of the initial Samsung ARMv5 tree I forked over two years ago. Of course, EmbeddedXen and ARM PV efforts haven't been sitting still in the meantime, but I'm treating the project as a personal "let's write a hypervisor" effort. There's no explicit desire to be compatible to any other ARM Xen effort.

My branch is targetting ARMv7, initially since that's what I've got around the house, but after digging through ARM ARM both for v6 and v7 I'm glad I'm not targeting anything older. The current differences from the original Samsung tree -
* ARMv7-only support.
* Tegra2 platform, targeting the Motorola Xoom
* Dom0/U configuration is not hardcoded.
* Boot through ATAG-compatible bootloader, with all images passed through a "boot volume".
* Kernel threads ("xen domains"), which are currently cooperative, but full preemption is a design goal.
* No ACM, given likely hypercall changes.
* A FIQ-based extensible serial kernel debugger with useful commands to help debugging and bring-up.
* In progress work on a framebuffer debug console.
* Reworked VMM and PT support with AFE and XN.

I'm concentrating on fleshing out crash debugging and framebuffer debug console support at the moment. After that it's (in no particular order) -
* L2 cache integration.
* Synchronization.
* Switching to VM and back.
* Full preemption for xen domains.
* Hypercall definitions.

I'll leave you with a short video demonstration of what I've got at the moment. It's not too amazing, the most interesting stuff is well ahead.

[0] http://labs.vmware.com/publications/2012-linux-symposium-publications
[1] https://github.com/andreiw/xen3-arm-tegra