Kernel hacking for fun, profit and blinkenlights

The hardware

About a month and a half ago, I saw an Xserve G4 going for cheap on eBay. Seeing as I already have a couple of bits of Apple PowerPC hardware of that vintage around, I decided to expand the collection a little. The specimen I received is a single-processor slot-load machine, which makes it one of the later G4's.

As with all other Xserves, there are sixteen LED's in two rows of eight on the front panel, and under Mac OS X, these LED's light up according to CPU utilisation. However, to match my existing machines, I installed OpenBSD instead, as the Xserve would be a far more capable build machine than my next most powerful machine, an iBook G4. OpenBSD has its own driver for the front panel LED's, namely xlights(4).

The blinkenlights

In order to understand how this works, I read both the manual page and the driver source code, and it turns out that the interface to the front panel LED's is a little interesting. The LED's are exposed to the operating system as a virtual I2S sound card, and when the xlights(4) driver is loaded, it reserves a segment of memory from which the sound card can do DMA reads. The driver then writes a stream of 32-bit integers into this segment of memory (where the bits set in each integer indicate which LED should be lit), and then tells the sound card to do the DMA read. This whole process is then wrapped in a loop to continuously feed the sound card with data. From here, it's pretty easy to do pulse width modulation to get the LED's to appear at different brightnesses.

The xlights(4) driver prints a funky little walking dots pattern, whose speed is inversely proportional to the system load. (Also, if the kernel crashes, then all the LED's will go out, as there's nothing to feed them any data, which is a useful visual indicator.) However, for the sake of novelty, I wanted to be able to turn the LED's on and off on demand from userspace, so I decided to write a driver to enable me to do this.

The hacking

When xlights(4) is initialised and finds a matching hardware device, it allocates the DMA space for the sound card, and then starts a kernel thread for generating the data to make the lights go on. This thread sits in the xlights_loop() function, which writes data into the DMA space and then prods the hardware to perform the DMA read. So, what I needed to do was to modify the kernel loop, and then add a userland interface.

First off, I went into /sys/arch/macppc/dev and copied xlights.c to xblink.c (as I couldn't think of a better name at the time) and then started tearing out the bits I didn't need. I reduced the main loop in the driver to simply set the LED levels based on the numbers in an array instead of calculating them on every iteration.

I decided I wanted to expose the LED's to userland through a device file on which one calls ioctl()s. Implementing this turns out to be a little complicated.

Internally, the OpenBSD kernel has a lists of character and block devices indexed by device major number which point to driver functions for handling filesystem operations which can be performed on those special devices. For the PowerPC kernel, these lists are defined by a load of macros in /sys/arch/macppc/macppc/conf.c. When a userland process opens a device file, the kernel virtual filesystem layer then looks up the driver for handling the device with the given major number, and then calls that driver's open() function, if it exists.

For my purposes, that meant defining defining open(), close() and ioctl() functions for the xblink(4) driver, which establish some book-keeping state (e.g. only one process can have the device file open at a time), and I added a new header file, /sys/arch/macppc/include/xblink.h to define the ioctl() constants and the struct which is passed in the ioctl() call. I then registered the xblink(4) functions in the list of kernel character devices, and added the appropriate entry to /sys/arch/macppc/conf/files.macppc to make the driver visible to the build system.

All that was left was to tweak my kernel configuration, rebuild the kernel, create the appropriate device file and then write a utility to make the ioctl() call.

The results

Much to my satisfaction, I eventually managed to get this to work, and I was able to toggle the LED's from userland. The only thing which I couldn't get to work was the fact that there was a noticable delay between making the ioctl() call and the front panel LED's changing, however I suspect this might have something to do with the frequency at which I was making the sound card poll the DMA segment.

It also turns out that this little adventure into kernel hacking was an excellent way to take my mind off the really nasty cold I had at the time, which was a much-appreciated reprieve.

Addendum: dmesg(8) output

(This isn't the dmesg(8) from the xblink(4) kernel, but it's a single line difference.)

OpenBSD 6.4 (GENERIC) #231: Thu Oct 11 17:55:03 MDT 2018
real mem = 1073741824 (1024MB)
avail mem = 1027231744 (979MB)
mpath0 at root                            
scsibus0 at mpath0: 256 targets           
mainbus0 at root: model RackMac1,2        
cpu0 at mainbus0: 7455 (Revision 0x303): 1333 MHz: 256KB L2 cache, 2MB L3 cache
mem0 at mainbus0                          
spdmem0 at mem0: 1GB DDR SDRAM non-parity PC3200CL3.0
memc0 at mainbus0: uni-n rev 0x24
kiic0 at memc0 offset 0xf8001000           
iic0 at kiic0                       
lmenv0 at iic0 addr 0xad: lm87 rev 6
lmtemp0 at iic0 addr 0x49: ds1775                                                
"cy2213" at iic0 addr 0x65 not configured  
mpcpcibr0 at mainbus0 pci: uni-north                                    
pci0 at mpcpcibr0 bus 0                                                 
bge0 at pci0 dev 16 function 0 "Broadcom BCM5703X" rev 0x02, BCM5702/5703 A2 (0x1002): irq 48, address 00:03:93:f4:44:e6
brgphy0 at bge0 phy 1: BCM5703 10/100/1000baseT PHY, rev. 2
mpcpcibr1 at mainbus0 pci: uni-north
pci1 at mpcpcibr1 bus 0
ppb0 at pci1 dev 13 function 0 "Intel 21154AE/BE" rev 0x00
pci2 at ppb0 bus 1
macobio0 at pci2 dev 7 function 0 "Apple Keylargo" rev 0x03
openpic0 at macobio0 offset 0x40000: version 0x4614 feature 3f0302 LE
"pwm" at macobio0 offset 0x30 not configured
macgpio0 at macobio0 offset 0x50
macgpio1 at macgpio0 offset 0x9: irq 47
"programmer-switch" at macgpio0 offset 0x11 not configured
"ringDetect-gpio" at macgpio0 offset 0x8 not configured
"keySwitch-gpio" at macgpio0 offset 0xc not configured
"systemMonitor-gpio" at macgpio0 offset 0x12 not configured
sysbutton0 at macgpio0 offset 0x15: irq 59
"fan0-gpio" at macgpio0 offset 0x1b not configured
"fan1-gpio" at macgpio0 offset 0x1c not configured
"indicatorLED-gpio" at macgpio0 offset 0x20 not configured
"virtual-sound" at macgpio0 not configured
"escc-legacy" at macobio0 offset 0x12000 not configured
zs0 at macobio0 offset 0x13000: irq 22,23
zstty0 at zs0 channel 0: console
zstty1 at zs0 channel 1
xlights0 at macobio0 offset 0x10000: irq 1
"timer" at macobio0 offset 0x15000 not configured
adb0 at macobio0 offset 0x16000
apm0 at adb0: battery flags 0x9, 0% charged
piic0 at adb0
iic1 at piic0
"PCA9554" at iic1 addr 0xa0 not configured
"PCA9554" at iic1 addr 0xa1 not configured
"PCA9554" at iic1 addr 0xa2 not configured
"PCA9554" at iic1 addr 0xa3 not configured
"PCA9554" at iic1 addr 0xa4 not configured
kiic1 at macobio0 offset 0x18000
iic2 at kiic1
wdc0 at macobio0 offset 0x1f000 irq 19: DMA   
atapiscsi0 at wdc0 channel 0 drive 0
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <QSI, CD-ROM TCR-241, WL11> ATAPI 5/cdrom removable
cd0(wdc0:0:0): using PIO mode 4, DMA mode 2
ohci0 at pci2 dev 8 function 0 "Apple USB" rev 0x00: irq 27, version 1.0
ohci1 at pci2 dev 9 function 0 "Apple USB" rev 0x00: irq 28, version 1.0
usb0 at ohci0: USB revision 1.0
uhub0 at usb0 configuration 1 interface 0 "Apple OHCI root hub" rev 1.00/1.00 addr 1
usb1 at ohci1: USB revision 1.0   
uhub1 at usb1 configuration 1 interface 0 "Apple OHCI root hub" rev 1.00/1.00 addr 1
ppb1 at pci1 dev 17 function 0 "Intel 21154AE/BE" rev 0x00 
pci3 at ppb1 bus 2
pciide0 at pci1 dev 21 function 0 "Promise PDC20271" rev 0x03: DMA, channel 0 configured to native-PCI, channel 1 configured to native-PCI
pciide0: using irq 58 for native-PCI interrupt
wd0 at pciide0 channel 0 drive 0: <ST3500830A>
wd0: 16-sector PIO, LBA48, 476940MB, 976773168 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5
pciide1 at pci1 dev 27 function 0 "Promise PDC20271" rev 0x03: DMA, channel 0 configured to native-PCI, channel 1 configured to native-PCI
pciide1: using irq 63 for native-PCI interrupt
mpcpcibr2 at mainbus0 pci: uni-north
pci4 at mpcpcibr2 bus 0
"Apple UniNorth Firewire" rev 0x01 at pci4 dev 14 function 0 not configured
gem0 at pci4 dev 15 function 0 "Apple Uni-N2 GMAC" rev 0x00: irq 41, address 00:0a:95:e6:e0:9e
brgphy1 at gem0 phy 0: BCM5421 10/100/1000baseT PHY, rev. 1
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
bootpath: /pci@f2000000/AppleKiwi@15/ata-6@0/disk@0:/bsd
root on wd0a (6687538f51f33579.a) swap on wd0b dump on wd0b