Porting UEFI to XXX, step 1

I've decided to do the actual blogging for this project *in* the repo itself. See https://github.com/andreiw/ppcnw-edk2/blob/master/README.md. After all, markdown is convenient enough and using Blogger on the G4 is p-a-i-n-f-u-l.

So it turns out that blogging about something after the fact is pretty tough. I really wanted to blog about my PoC port of UEFI to the OpenPower ecosystem, but it's incredibly difficult to go back and try to systematize something that's been a few years back.

So let's try this again. This time, our victim will be a G4 12" PowerBook6,8 with a 7447A. That's a 32-bit PowerPC. Now, I'll go in small steps and document *everything*. For added fun, we'll begin porting on the target itself, at least until that gets too tedious.

First, I updated to the latest (and last) Debian 8 (Jessie).

Now let's clone the tree.

$ git clone https://github.com/tianocore/edk2

Setup the UEFI environment.

$ cd edk2

$ . edksetup.sh

Now we need to get the BaseTools building.

pbg4:~/src/edk2/BaseTools/ make

make -C Source/C

make[1]: Entering directory '/home/andreiw/src/edk2/BaseTools/Source/C'

Attempting to detect ARCH from 'uname -m': ppc

Could not detected ARCH from uname results

GNUmakefile:36: *** ARCH is not defined!.  Stop.

make[1]: Leaving directory '/home/andreiw/src/edk2/BaseTools/Source/C'

GNUmakefile:25: recipe for target 'Source/C' failed

make: *** [Source/C] Error 2

Ok. Let's fix that. We'll first need a Source/C/Include/PPC/ProcessorBind.h file.

ProcessorBind.h I've derived from another 32-bit CPU, like IA32 or ARM. This contains type definitions, mostly. It's boilerplate. In case there are multiple coding conventions for your architectures and it's not obvious which one you should be using, you might wish to specify what the EFIAPI attribute will be. Like, on x86 Windows-style cdecl is used, regardless of how you build the rest of Tiano. On most architectures an empty define is fine.

Now appropriately hook it into Source/C/Makefiles/header.makefile.

--- a/BaseTools/Source/C/Makefiles/header.makefile

+++ b/BaseTools/Source/C/Makefiles/header.makefile

@@ -43,6 +43,10 @@ ifeq ($(ARCH), AARCH64)

 ARCH_INCLUDE = -I $(MAKEROOT)/Include/AArch64/

 endif

+ifeq ($(ARCH), PPC)

+ARCH_INCLUDE = -I $(MAKEROOT)/Include/PPC/

+endif

Fix the ARCH detection in Source/C/GNUmakefile.

--- a/BaseTools/Source/C/GNUmakefile

+++ b/BaseTools/Source/C/GNUmakefile

@@ -31,6 +31,9 @@ ifndef ARCH

   ifneq (,$(findstring arm,$(uname_m)))

     ARCH=ARM

   endif

  ifneq (,$(findstring ppc,$(uname_m)))

    ARCH=PPC

  endif

Ok, ensure you have the libuuid headers (Debian uuid-dev) and g++. And...

You are done. This gives you the tools need to help build UEFI. Now we need to teach the build system about PowerPC...

Toying around with LE PowerPC64 via the Power8 simulator

ppc64le_hello is simple example of what it takes to write stand-alone (that is, system or OS) code that runs in Little-Endian and Hypervisor modes on the latest OpenPOWER/Power8 chips. Of course, I don't have a spare $3k to get one of these nice Tyan reference systems, but IBM does have a free, albeit glacially slow and non-OSS, POWER8 Functional Simulator.

What you get is a simple payload you can boot via skiboot, or another OPAL-compatible firmware. Features, in no particular order:

• 64-bit real-mode HV LE operation.
• logging via sim inteface (mambo_write).
• logging via OPAL firmware (opal_write).
• calling C code, stack/BSS/linkage setup/TOC.
• calling BE code from LE.
• FDT parsing, dumping FDT.
• Taking and returning from exceptions, handling unrecoverable/nested exceptions.
• Timebase (i.e. the "timestamp counter"), decrementer and hypervisor decrementer manipulation with some basic timer support (done for periodic callbacks into OPAL).
• Running at HV alias addresses (loaded at 0x00000000200XXXXX, linked at 0x80000000200XXXXX). The idea being that the code will access physical RAM and its own data structures solely using the HV addresses.
• SLB setup: demonstrates 1T segments with 4K base page and 16M base page size. One segment (slot = 0) is used  to back the HV alias addresses with 16M pages. Another  segment maps EA to VA 1:1 using 4K pages.
• Very basic HTAB setup. Mapping and unmapping for pages in the 4K and 16M segments, supporting MPSS (16M pages in the 4K segment). No secondary PTEG. No eviction support. Not SMP safe. Any access within the HV alias addresses get mapped in. Any faults to other  unmapped locations are crashes, as addresses below 0x8000000000000000 should only be explicit maps.
• Taking exception vectors with MMU on at the alternate vector location (AIL) 0xc000000000004000.
• Running unpriviledged code.

See README for more information, including how to build and run. At some point it ran on a real Power8 machine - and may run still ;-).

iQUIK supports the Performa 6400

After fixing some sad bugs from the last refactoring binge and adding support for OF 2.0, the 6400 (and likely all "Alchemy"-based Macs) can be booted via iQUIK.

Just like OpenFirmware 2.0.1, 2.0 seems to suffer from the "shallow setprop" bug, that results in bogus values for the /chosen/linux,initrd-start and /chosen/linux,initrd-end properties.

Solaris/PPC

Apparently back around 2006 there was an effort at Sun Labs to get OpenSolaris to work on CHRP(like)  PowerPC machines. And according to the documentation, the kernel could even boot to shell on a G4 Apple.

That effort was called Polaris. It was difficult to find the CDDL-licensed sources, but I've made them available for everyone else to play with at https://github.com/andreiw/polaris

I haven't tried it out or done anything with the sources yet. The Solaris kernel is a pretty amazing piece of software, and a very portable and well-designed one to boot. I am glad Sun open-sourced it before folding, as it's code like this that should be influencing OS R&D for generations to come. It would be interesting to see the Polaris code being used as a base for an AArch64 investigation...

A

What's special about...

addi r0,r1,0x138
ori r0,r0,0x60

...and I/O port 0x92 :-)?

iQUIK update

I now have a 1.5Ghz PowerBook 12" in my possession to test iQUIK with. This is of course a NewWorld, and not a primary target for the iQUIK boot loader...

Couple of observations to be made:

• OpenFirmware 3.0 doesn't support partition zero booting (i.e. hd:0 or CHRP-spec hd:%BOOT). This means that iQUIK cannot be booted the same way as it boots on OldWorlds,  but neither is it required. iQUIK can be booted on NewWorlds the same way as Yaboot, i.e. placing 'iquik.elf' on an HFS+ partition and blessing it. 
• NewWorld OF requires appending ":0" for full-disk access to disk devices

I've also fixed a bug inside partition code that truncated offsets to 32 bits, and improved device path handling and parsing.

In short, though, it works. And it works quite well. So iQUIK now works on OldWorld and NewWorld machines. Yaboot - only on NewWorlds. Of course, Yaboot also supports CHRP machines, network booting and reads all filesystems supported by the underlying OpenFirmware implementation. So there's plenty of work to reach feature parity in that regard.

A

Making OpenBIOS support Apple partition-zero booting

To speed up my development cycle on iQUIK and make it less painful I really needed to get OpenBIOS to boot via bootcode correctly. The current sources (r1272) hard code the load address for the legacy QUIK bootloader, which makes it useless for me or for anyone else (like the NetBSD or OpenBSD bootloaders).

I didn't try very hard, but the end result works, and hopefully the OpenBIOS guys will just take my fix.

SVN workflow seems so...senescent compared to git. Sigh.

Index: forth/debugging/client.fs
===================================================================
--- forth/debugging/client.fs (revision 1272)
+++ forth/debugging/client.fs (working copy)
@@ -28,7 +28,13 @@
 0 state-valid !
 
 variable want-bootcode
+variable bootcode-base
+variable bootcode-size
+variable bootcode-entry
 0 want-bootcode !
+0 bootcode-base !
+0 bootcode-size !
+0 bootcode-entry !
 
 variable file-size
 
Index: libopenbios/bootcode_load.c
===================================================================
--- libopenbios/bootcode_load.c (revision 1272)
+++ libopenbios/bootcode_load.c (working copy)
@@ -12,13 +12,11 @@
 #define printf printk
 #define debug printk
 
-#define OLDWORLD_BOOTCODE_BASEADDR (0x3f4000)
-
 int 
 bootcode_load(ihandle_t dev)
 {
     int retval = -1, count = 0, fd;
-    unsigned long bootcode, loadbase, offset;
+    unsigned long bootcode, loadbase, offset, loadsize, entry;
 
     /* Mark the saved-program-state as invalid */
     feval("0 state-valid !");
@@ -33,34 +31,59 @@
     loadbase = POP();
     
 #ifdef CONFIG_PPC
-    /* ...except that QUIK (the only known user of %BOOT to date) is built
-       with a hard-coded address of 0x3f4000. Let's just use this for the
-       moment on both New World and Old World Macs, allowing QUIK to also
-       work under a New World Mac. If we find another user of %BOOT we can
-       rethink this later. PReP machines should be left unaffected. */
+    /*
+     * Apple OF does not honor load-base and instead uses pmBootLoad
+     * value from the boot partition descriptor.
+     *
+     * Tested with:
+     *   a debian image with QUIK installed
+     *   a debian image with iQUIK installed (https://github.com/andreiw/quik)
+     *   an IQUIK boot floppy
+     *   a NetBSD boot floppy (boots stage 2)
+     */
     if (is_apple()) {
-        loadbase = OLDWORLD_BOOTCODE_BASEADDR;
+      feval("bootcode-base @");
+      loadbase = POP();
+      feval("bootcode-size @");
+      loadsize = POP();
+      feval("bootcode-entry @");
+      entry = POP();
+
+      printk("bootcode base 0x%lx, size 0x%lx, entry 0x%lx\n",
+             loadbase, loadsize, entry);
+    } else {
+      entry = loadbase;
+
+      /* Load as much as we can. */
+      loadsize = 0;
     }
 #endif
     
     bootcode = loadbase;
     offset = 0;
     
-    while(1) {
+    if (loadsize) {
+      if (seek_io(fd, offset) != -1)
+        count = read_io(fd, (void *) bootcode, loadsize);
+    } else {
+      while(1) {
         if (seek_io(fd, offset) == -1)
-            break;
+          break;
         count = read_io(fd, (void *)bootcode, 512);
         offset += count;
         bootcode += count;
+      }
     }
 
     /* If we didn't read anything then exit */
     if (!count) {
         goto out;
     }
+
+    printk("entry = 0x%lx\n", entry);
     
     /* Initialise saved-program-state */
-    PUSH(loadbase);
+    PUSH(entry);
     feval("saved-program-state >sps.entry !");
     PUSH(offset);
     feval("saved-program-state >sps.file-size !");
Index: libopenbios/load.c
===================================================================
--- libopenbios/load.c (revision 1272)
+++ libopenbios/load.c (working copy)
@@ -1,6 +1,6 @@
 /*
  *   Creation Date: <2010/06/25 20:00:00 mcayland>
- *   Time-stamp: <2010/06/25 20:00:00 mcayland>
+ *   Time-stamp: <2014-03-05 03:18:49 andreiw>
  *
  * <load.c>
  *
Index: packages/mac-parts.c
===================================================================
--- packages/mac-parts.c (revision 1272)
+++ packages/mac-parts.c (working copy)
@@ -1,6 +1,6 @@
 /*
  *   Creation Date: <2003/12/04 17:07:05 samuel>
- *   Time-stamp: <2004/01/07 19:36:09 samuel>
+ *   Time-stamp: <2014-03-05 03:42:07 andreiw>
  *
  * <mac-parts.c>
  *
@@ -237,6 +237,19 @@
      size = (long long)__be32_to_cpu(par.pmPartBlkCnt) * bs; 
      
      if (want_bootcode) {
+  ucell loadaddr = 0;
+  ucell loadsize = 0;
+  ucell loadentry = 0;
+
+  loadaddr = __be32_to_cpu(par.pmBootLoad);
+  loadsize = __be32_to_cpu(par.pmBootSize);
+  loadentry = __be32_to_cpu(par.pmBootEntry);
+  PUSH(loadaddr);
+  feval("bootcode-base !");
+  PUSH(loadsize);
+  feval("bootcode-size !");
+  PUSH(loadentry);
+  feval("bootcode-entry !");
   offs += (long long)__be32_to_cpu(par.pmLgBootStart) * bs;
   size = (long long)__be32_to_cpu(par.pmBootSize);
      }
@@ -249,6 +262,10 @@
      di->size_hi = size >> BITS;
      di->size_lo = size & (ucell) -1;
 
+     if (want_bootcode) {
+       goto out;
+     }
+
      /* We have a valid partition - so probe for a filesystem at the current offset */
      DPRINTF("mac-parts: about to probe for fs\n");
      DPUSH( offs );
@@ -277,7 +294,7 @@
   
       /* If we have been asked to open a particular file, interpose the filesystem package with 
       the passed filename as an argument */
-      if (!want_bootcode && strlen(argstr)) {
+      if ( strlen(argstr)) {
        push_str( argstr );
        PUSH_ph( ph );
        fword("interpose");
@@ -286,6 +303,10 @@
       goto out;
      } else {
       DPRINTF("mac-parts: no filesystem found on partition %d; bypassing misc-files interpose\n", parnum);
+
+      /* Fail out instead of having macparts_load get called uselessly, allowing trying the next
+         boot device */
+      ret = 0;
      }
  }

This does make booting iQUIK on OpenBIOS now very easy.

qemu-system-ppc -hda ~/src/quik/distrib/floppy-cfg.img -prom-env "boot-file=hd:3"

Getting PowerPC OpenBIOS to run on QEMU

• You've apt-get installed qemu, but qemu-system-ppc boots to a blank (white or black) screen?
• You've pulled the OpenBIOS SVN, built qemu-openbios.elf, but it boots to a blank screen?

On serial output, you might see "<< set_property: NULL phandle" messages, and the CPU is stuck in a perpetual ISI.

Have no fear. Apparently GCC versions > 4.6 miscompile OpenBIOS, so you need to disable optimization. This is presently set to "-Os" under "Makefile.target". Setting it t "-O0" should do.

I'll probably investigate this deeper after fixing partition-zero booting...

A

Working around OFW and kernel bugs on OldWorlds

Trying to boot up a 2.6 kernel with initrd on the 3400c I ran into a couple of bugs that were show stoppers. One resulted in the kernel not finding it's initrd image ("VFS panic"), the second in a hard hang probing a non-existing media bay ATA.

To understand the first bug (and the workaround for both) we need to see how the OFW structures are used in kernel bootup.

On PowerPC, the Linux kernel can be called anywhere. The early boot code is capable of running at any address (it is reloc-safe), but the kernel needs to be moved to its linked address before it can be run. Of course, the linked address might conflict with OFW mappings, hence the early boot code needs to exit the firmware services before relocating and jumping to the rest of the kernel. Of course, the rest of the kernel needs the device tree information to initialize the platform, and because OFW has been already quiesced, the early boot code needs to make a copy of the device tree structures, thereby "flattening" it. This early OFW-handling code is called from kernel/head_32.S and is in arch/powerpc/kernel/prom_init.c(prom_init).

Before flattening the device tree, prom_init first queries various bits and pieces and uses the OF setprop call to store them inside the /chosen node. The /chosen node represents information passed from OFW to the OS, and Linux sets properties in that node for itself to find later. For example, the initrd information (base and size), which is passed in registers to the kernel from the bootloader, is then stored in /chosen/linux,initrd-start and /chosen/linux,initrd-end. This information is then copied using the OFW getprop call when the dt is flattened later inside prom_init.

The problem on the 3400c with OFW 2.0.1 is that the setprop client call doesn't actually make a copy of the data, but just stashes the passed pointer away. Because the values set during calls from prom_init are built on the stack, the later getprop calls during dt flattening return garbage.

So how can we deal with it? Because we know the kernel only makes calls to the PROM before it trashes the memory map, we can "shim away" the real client interface with a filter inside the bootloader, that can
feed the kernel the right values and fix up any broken OF behavior. In case of the "shallow" setprop problem above, I simply feed the correct initrd base and end addresses when I detect accesses to /chosen/linux,initrd-start and /chosen/linux,initrd-end. Of course, a much better and generic way would be to make deep copies of any setprop-ed field onto the bootloader heap...

Come to think of it, it's highly likely older OF versions (like the infamous 1.0.5) have the same bug.

The second bug involved a hard hang scanning the media bay ATA controller on the 3400c. I don't have a CD drive, so I have no idea if this only happens with the bay empty. There are no kernel parameters to black list devices. Since we already have a shim to intercept and transform OF client interface results, I simply have hidden /bandit/ohare/media-bay/ata away from the kernel.

iQUIK - the new old PowerMac bootloader for OldWorld machines

About three quarters of a year ago I found a PowerBook 3400c in the trash pile at work. Given my ongoing nostalgia for PowerPC and OpenFirmware, I couldn't pass up the opportunity to reminisce with this PowerBook.

It is by far the oldest mac I've had. The UMAX S900 I dumpster dove for back in 2002 was a dual 604e. This baby is a 603ev. That's three generations behind my old G4 iBook. It's OF is slightly less broken than 1.0.5 on the S900 - at least it doesn't default stdio to serial, and doesn't need nvramc patches to boot from disk. The 3400c booted into MacOS, but clearly it was asking for Linux.

Unfortunately, the best you could do is install via old miBoot floppies, and get a 2.2(!)-based Debian system, just like I had to on the S900 ten whole years ago. Of course you could painfully and slowly upgrade releases, only to realize you can't actually boot a 2.4 kernel with an initrd with the OldWorld bootloader, QUIK. Staring at a "VFS panic" message after investing a week into slowly getting to /that/ point would be bound to enrage most people just toss the aging laptop into trash...

I guess I have an odd fascination with obsolete hardware. I've decided to take ownership of the QUIK code, fixing many of its bugs and limitations. The effort is now called iQUIK and is available for everybody at https://github.com/andreiw/quik.

iQUIK is a Linux bootloader for "OldWorld" PowerMacs. "OldWorld" PowerMacs are all machines that have OpenFirmware, but that don't have built-in USB. Due to a lacking firmware implementation, the typical Yaboot bootloader cannot be used on "OldWorld" machines.

Some highlights:

• You can install and boot Debian Wheezy (the current stable release)
• You can install the boot loader on any medium, including floppy.
• Initrd image work fine, as do 2.2., 2.4 and 2.6 kernels.
• Can boot any kernel/initrd/argument combos without a quik.conf.
• Can list filesystems (only ext2 supported at the moment).
• Works around OF/hw bugs with an innovative shim layer. At least the many bugs on the 3400c.
• Single-stage installation for robustness, using partition-zero boot block and bootstrap partition.No more LILO-style block maps.

You can see the fairly exhaustive documentation at https://github.com/andreiw/quik/blob/master/README. It's only really been tested on a 3400c, although I'm getting a G3 PDQ (that's the last OldWorld PowerBook!) for testing soon.

...but that's not what I wanted to write this post about. I was curious if I could boot iQUIK successfully with the QEMU PowerPC system emulator and OpenBIOS. Turns out yes, but it's a bit painful. The problem is that OpenBIOS doesn't support the mac's "partition-zero" booting, e.g.

0 > boot ata0/ata-disk@0:0

How does "partition-zero" booting work? The mac partition table contains a structure that describes the disk offset of the bootloader, its size and where to load it. The bootloader has to be plain binary (i.e. not XCOFF). iQUIK leverages this by copying the boot code into it's own partition and setting up the boot descriptor in the partition table to point to it. http://www.opensource.apple.com/source/bless/bless-11/README.BOOTING is a pretty good document that describes this more in-depth.

So how to boot on QEMU, then? We could cheat and load the non-munged ELF file the boot code is created from, but I don't have my 3400c nearby to compile with. Instead we will convince OpenBIOS to load our code just like a PowerMac would.

This example assumes:

• You have qemu-system-ppc.
• You have floppy.img from the github quik/distrib.
• You created an install.img: FAT partitioned, ext2 on only FAT partition with the Debian files from http://ftp.nl.debian.org/debian/dists/wheezy/main/installer-powerpc/current/images/powerpc/hd-media/
• Note: take the 'initrd-size' out of yaboot.conf file you downloaded above.

Steps are pretty simple. We launch the simulator like so:

qemu-system-ppc  -hda floppy.img -hdb install.img -serial stdio

At the "0 >" prompt in the framebuffer window we type the following commands to switch the firmware to serial port for input/output. You might wonder why we couldn't use the "prom-env" option to set output-device and input-device? It doesn't work. Heh.

0 > " /pci/mac-io/escc/ch-a" output
0 > " /pci/mac-io/escc/ch-a" input

And now paste the rest of this script into terminal you ran QEMU from. Of course, you could put it on bootable media, but...

0 value ih
\
\ size and load-base need to match
\ SECOND_BASE and SECOND_SIZE in
\ quik/include/layout
\
10000 value size
setenv load-base 3e0000
\
\ boot-file specifies where iQUIK will look
\ for the configuration file. This corresponds to
\ the second ATA disk (-hdb), first FAT partition.
\
setenv boot-file /pci/mac-io/ata-1/disk@1:1/yaboot.conf
\
\ OpenBIOS doesn't know how to parse the MAC "partition-zero"
\ boot descriptor, so we'll manually load it. XXX:2 refers
\ to the Apple_Boostrap partition on -hda, which in the floppy.img
\ image is the first parition (wonky indexing... yes).
\
\ I don't really understand why OpenBIOS needs the seek to 0.
\ Seems like a bug...
\
" /pci/mac-io/ata-1/disk@0:2" open-dev to ih
0 " seek" ih $call-method .
load-base size " read" ih $call-method .
\
\ Based on OpenBIOS libopenbios/bootcode_load.c
\
load-base saved-program-state >sps.entry !
size saved-program-state >sps.file-size !
bootcode saved-program-state >sps.file-type !
-1 state-valid !
\
\ Now can boot. For completeness probably should set
\ /chosen/bootargs but iQUIK can handle the inconsistency.
\
ih close-dev
go

You should now see iQUIK in the framebuffer window (not the serial, because we haven't set the input-device and output-device env variables). You can press the tab key to show the boot labels available. Just press enter to boot the default 'install'.

There you have it.

Of course it would be ideal to just implement bootsector-zero support or even this hack as an nvramrc patch, but unfortunately QEMU PPC has no NVRAM support. Boo...