common tasks

Run all the code coverage tests.

./gradlew testCodeCoverageReport

Load the code coverage reports into the browser

npx http-server gradle/aggregation/build/reports/jacoco/testCodeCoverageReport/html
# or
open gradle/aggregation/build/reports/jacoco/testCodeCoverageReport/html/index.html

Run the quality gate to do any necessary reformatting

./gradlew qualityGate

If you have trouble with CI failing but your local build isn't, try updating from main and then run a clean build locally with:

 ./gradlew clean --no-daemon --no-build-cache --no-configuration-cache
 ./gradlew checkAllModuleInfo --no-daemon --no-build-cache --no-configuration-cache 

Before committing make sure

• GPG signing is set up
• commit sign-off is set up
• ./gradlew testCodeCoverageReport passes
• ./gradlew qualityGate passes

If GPG signing was working and stops suddenly, it's probably because the ssh-agent has stopped. Run ssh-agent to restart it.

For embedded development, once Rust itself is installed, you'll need to install the esp toolchain with espup, then use it to install the esp toolchains. This will take a while.

# do this somewhere other than a rust project dir.
cargo install espup --locked
# add .cargo/bin to your path
export PATH="$PATH:/Users/username/.cargo/bin"

# then install the esp toolchains. This will take a while.
espup install
# source the env vars
. ~/export-esp.sh

# tool to generate new esp projects
cargo install esp-generate
# tool to flash the attached device
cargo install espflash --locked

# make a new esp project
esp-generate --chip=esp32s3 your-project

Leave the default options, hit 's', cd into the project, and build it! Make sure to enable 'log' so you can do the helloworld.

cd your-project
cargo build
cargo run

if you get a waiting for download message, press the RST button and run it again. Then you should see hello world printed.

switch power off press and hold the trackball switch power on have 2 seconds release the trackball

Also needed ESP-IDF

brew install cmake ninja dfu-util

T-Deck with Rust

I'm using the latest esp-hal, which is 1.0.0-beta.1 at the time of this writing (June 10st 2025) I can get the blinky example to compile, but since the t-deck doesn't have a user accessible GPIO pin that doesn't do anything.

I created a repo with examples for the Tdeck showing how to set up the display, connect to wifi, access the keyboard and trackball, and read the battery level.

To install a tool via cargo do cargo install. To install it from a specific version from git do:

cargo install espflash --git https://github.com/JurajSadel/espflash.git

To choose a branch do

cargo install espflash --git https://github.com/JurajSadel/espflash.git --branch feat/reset-args

JoshMcguigan/cargo-run-script: Bringing npm run-script to Rust

• create ssh keypair on the server as a specific user used just for uploading
• copy the private key to a github secret
• append the public key to the .ssh/authorized_keys file
• make a github action that uploads the artifacts using this scp action

The action looks like this:

name: Publish Brain

on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
      - run: npm ci
      - run: npm run build-brain
      - name: copy file with ssh password
        uses: appleboy/scp-action@v0.1.7
        with:
          host: myserver.com
          username: deployuser
          port: 22
          key: ${{secrets.DEPLOYUSER_PRIVATE_KEY}}
          source: "*.html"
          target: remote_path

So you make a commit and pushed it to GitHub, then realized you forgot to sign it. fix it with git rebase HEAD~N --signoff. where N is the number of recent commits you need to sign.

git pull
git rebase HEAD~1 --signoff
git push --force

This means a branch of a branch, which is useful when you want to work on a second feature depending on the first PR, while the first PR is still being reviewed.

# create a branch
git branch b1
git checkout b1
# do your code changes and commit them
# create a second branch
git branch b2
git checkout b2
# do your code changes and commit them
# print the branch status
git branch -vv
# make sure the first branch is pushed to github (or whatever your origin)
git push -u origin b1
# make sure the second branch is pushed to github (or whatever your origin)
git push -u origin b2
# this will print a link to create a PR on github.

If you have two local copies of a repo, you can add one as a remote for the other, to push branches between them.

First, add the second dir as a remote for the first, giving it a name (josh_fork in this case).

git remote add josh_fork ../hiero-consensus-node-personal

Then push to that local remote with -u.

git push -u josh_fork

checkout a branch

git checkout branchname

force git to forget about a file

git rm --cached .gitbook/assets/Jenkins.png

to restore a file to what it was before current working changes

git restore file

to abort a rebase operation

git rebase --abort

show the remote url

git remote show origin

show the status of your branches

git branch -vv

button {
    --red: #ff0000;
    /* keep hue and saturation, multiply l times 1.5 */
    --light-red: hsl(from var(--red) h s calc(l*1.5));
}

The logger framework lets you set log levels and send your logs to multiple outputs. There is a FileHandler class that will log to a file, however it does not flush automatically. I’m not sure when it ever flushes, actually, which is a problem for long running processes that only log occasionally. so I created my own JoshFileHandler that flushes after every log.

visual logging

terminalio – Displays text in a TileGrid — Adafruit CircuitPython 8.2.0-rc.0 documentation

Using plain terminalio with displayio - ValueError: Tile index out of bounds · Issue #7885 · adafruit/circuitpython · GitHub

Get the time

import time
now = time.localtime()
month = now[1]
day = now[2]
(hour, minutes, seconds) = now[3:6]

print datetime in isoformat

timestamp.isoformat()

The terminalio.Terminal class implements a traditional 'terminal' with VT100 control codes. It does this by giving you a stream you can print to. It converts the stream of characters into a tilegrid using the bitmap font. It will wrap lines if necessary. Unfortunately it only supports a single color at a time. Multi-colored text isn't supported.

import terminalio
import displayio

# size of each font glyph (this is a monospaced font)
fontx, fonty = terminalio.FONT.get_bounding_box()

# adjust palette if desired
plain_palette = displayio.Palette(2)
plain_palette[0] = 0x000000
plain_palette[1] = 0x33ff33

# create grid using the font 
# width and height in tiles
# x and y in pixels
pgrid = displayio.TileGrid(terminalio.FONT.bitmap,
                           x=0,
                           y=0,
                           width=display.width // fontx,
                           height=display.height // fonty,
                           tile_width=fontx,
                           tile_height=fonty,
                           pixel_shader=plain_palette)

# terminal attached to the tilegrid
ptermx = terminalio.Terminal(pgrid, terminalio.FONT)

# add to the screen
display.root_group.append(pgrid)

Print to the terminal with print:

# with newline
print("some text", file=ptermx, end="\r\n")

# without newline
print("some text", file=ptermx, end="")

E-ink screens show only black and white, but can effectively show grayscale using dithering. Internally the screens use a full color frame buffer, so colors should be specified the same as normal.

Fill the background with white using a single large bitmap

# fill the background with white
background_bitmap = displayio.Bitmap(WIDTH, HEIGHT, 1)
# Map colors in a palette
palette = displayio.Palette(1)
palette[0] = WHITE # 0xffffff
# Create a Tilegrid with the background and put in the displayio group
t = displayio.TileGrid(background_bitmap, pixel_shader=palette)
group.append(t)

Then set the color of labels to be BLACK as 0x000000.

A bitmap is just an empty image. It has a fixed size and number of colors. Pixels can be set directly using x,y coordinates.

import displayio


bitmap = displayio.Bitmap(32,32,2)
palette = displayio.Palette(2)
palette[0] = 0x000000
palette[1] = 0xffffff

# set pixel at x=3, y=4 to color 1
bitmap[3,4] = 1

Bitmaps can only be shown on screen using a tilegrid. To create a tilegrid that shows the bitmap without any repetition, do:

tilegrid = displayio.TileGrid(bitmap, pixel_shader=palette)
display.root_group.append(tilegrid)

convert a PNG to a indexed color bitmap

convert digits.png -colors 64 -type palette -compress None BMP3:digits.bmp

Use a custom font in a label or button

Overview | Custom Fonts for CircuitPython Displays | Adafruit Learning System

Convert truetype font to bitmap

brew install otf2bdf
otf2bdf FontFile.ttf -p pointsize -o FontFile.bdf

convert bitmap ascii to bitmap binary bdftopcf font converter

Use in python code

from adafruit_display_text import label
from adafruit_bitmap_font import bitmap_font

font = bitmap_font.load_font("my_font.bdf")
# draw in red
text_label = label.Label(font, text="Greetings Earthling!", color=0xFF0000)

create a new text label:

from adafruit_display_text import label
 
label = label.Label(
    font=terminalio.FONT,
    text="Greetings Earthlings",
    x=20,
    y=10,
    scale=1
)
display.root_group.append(label)

Everything graphics in CircuitPython is built around the displayio library.

• API docs
• Official Adafruit Tutorial

display graphics

First you need a display object, which is usually preconfigured for your board if it has a built in display, as board.display. It will automatically refresh the screen. The root of the screengraph should already be set to a group, so you can append to it.

display = board.display
display.root_group.append(some_gfx_object)

import unittest

class BasicParsing(unittest.TestCase):
    def test_thing(self):
        self.assertEqual("foo","bar")

if __name__ == '__main__':
    unittest.main()

Load JSON from disk

config = json.load(open("config.json",'rb'))
print(config['foo']['bar'])

read text file to string

with open("./blog.html", "r") as txt:
    html = txt.read()

list wifi

for network in wifi.radio.start_scanning_networks():
    print(f"{network.ssid} [Ch:{network.channel}] RSSI: {network.rssi}")

connect to wifi

wifi.radio.connect(ssid, password)

print wifi info

if wifi.radio.connected:
    print("wifi SSID info = ",wifi.radio.ap_info.ssid)
    print('transmit power', wifi.radio.tx_power)
    print(f"My MAC address: {[hex(i) for i in wifi.radio.mac_address]}")
    print("radio enabled", wifi.radio.enabled)
    print("host name", wifi.radio.hostname)

fetch HTTP request

# Initalize Wifi, Socket Pool, Request Session
pool = adafruit_connection_manager.get_radio_socketpool(wifi.radio)
ssl_context = adafruit_connection_manager.get_radio_ssl_context(wifi.radio)
requests = adafruit_requests.Session(pool, ssl_context)

with requests.get("http://webpage.site") as response:
    print(response.text)

CircuitPython Tricks

get the network time

# specific to the MatrixPortal M4
pool = adafruit_connection_manager.get_radio_socketpool(matrix.network._wifi.esp)
# most modern boards
pool = adafruit_connection_manager.get_radio_socketpool(wifi.radio),

# get the time
ntp = adafruit_ntp.NTP(socketpool=pool)

# print the time
print(ntp.datetime)

For the MagTag use magtag.get_local_time(), but you’ll need to setup the Adafruit IO credentials in settings.toml.

magtag.get_local_time() # sync with the network time
now = time.localtime()

RP2040 with USB Host

usb_host.Port(board.USB_HOST_DATA_PLUS, board.USB_HOST_DATA_MINUS)
if supervisor.runtime.usb_connected:
    print("USB<host>!")
else:
    print("!USB<host>")

while True:
    print(sys.stdin.read(1))

Scan for devices on USB host

while True:
    print("searching for devices")
    for device in usb.core.find(find_all=True):
        print("pid", hex(device.idProduct))
        print("vid", hex(device.idVendor))
        print("man", device.manufacturer)
        print("product", device.product)
        print("serial", device.serial_number)
        print("config[0]:")
        config_descriptor = adafruit_usb_host_descriptors.get_configuration_descriptor(
            device, 0
        )

        i = 0
        while i < len(config_descriptor):
            descriptor_len = config_descriptor[i]
            descriptor_type = config_descriptor[i + 1]
            if descriptor_type == adafruit_usb_host_descriptors.DESC_CONFIGURATION:
                config_value = config_descriptor[i + 5]
                print(f" value {config_value:d}")
            elif descriptor_type == adafruit_usb_host_descriptors.DESC_INTERFACE:
                interface_number = config_descriptor[i + 2]
                interface_class = config_descriptor[i + 5]
                interface_subclass = config_descriptor[i + 6]
                print(f" interface[{interface_number:d}]")
                print(
                    f"  class {interface_class:02x} subclass {interface_subclass:02x}"
                )
            elif descriptor_type == adafruit_usb_host_descriptors.DESC_ENDPOINT:
                endpoint_address = config_descriptor[i + 2]
                if endpoint_address & DIR_IN:
                    print(f"  IN {endpoint_address:02x}")
                else:
                    print(f"  OUT {endpoint_address:02x}")
            i += descriptor_len
        print()
    time.sleep(5)

• turn an exception into an array of strings with traceback.format_exception(e)
• format an array of strings into a single string with ”some string”.join(arr). 
• catch an exception:

    try:
        save_pixels('/screenshot.bmp',pixel_source=display)
        logger.info("saved the screenshot")
    except Exception as e:
        logger.error("couldnt take screenshot")
        logger.error(''.join(traceback.format_exception(e)))

• format a float: print(f”num = {num}”).

• format a float with leading zeros up to two chars wide: print(f”num = {num:02}”).

Also called the Sunton ESP32-2432S028. Like many ESP32 boards you must install CircuitPython using the web install using Chrome, or the commandline esp32 tool.

This little board is ESP based and doesn't come from Adafruit so you can't enter bootmode to open up a local drive. You have to install circuitpython over serial connection. The easiest way to do this is from the CP install page's new 'open installer button'. You must do this part on Chrome.

1. plug the device into your computer using a USB cable. make sure it's a data cable, not just power.
2. Hold the boot button, click the reset button, then release the boot button. It should be in bootmode. From the commandline on a a mac you should be able to see the serial port using ls -l /dev/tty.*.
3. Go to the CircuitPython release page using Chrome.
4. Click the 'open installer' button (on the right side below the purple download buttons). image here
5. Click the 'Upgrade CircuitPython 9.2.1 Bin Only' link in the dialog and follow the prompts. Assuming the device's serial port was found you should have CP installed after a minute or two.
6. Reboot the device and the CIRCUITPY drive should come up.

If your magtag already has the U2F bootloader on it, you can just double-click the reset button to enter bootmode, then drag the .uf2 file with the latest release of CircuitPython onto the magtag bootloader drive that appears. Otherwise you will need to use the ESP32 bin tool to install it.

The magtag has peripherals and here is the printout.

Init the MagTag object:

from adafruit_magtag.magtag import MagTag
magtag = MagTag()

The MagTag has a buzzer. Play tones with:

magtag.peripherals.play_tone(<frequency>,<seconds>)

Check the battery level by looking at the magtag.peripherals.battery property.

print("battery is", magtag.peripherals.battery)

Deep Sleep

The MagTag can enter a super deep sleep for a certain number of seconds using the exit_and_deep_sleep function. When the time is up the MagTag will reboot and run the contents of code.py again.

magtag.exit_and_deep_sleep(15*60) # 15 minutes

To do a lighter sleep and wake up rather than restarting use the enter_light_sleep function.

magtag.enter_light_sleep(60) # 60 seconds

Assuming the display is connected to the standard SPI connections on an RP2040

displayio.release_displays()
bus = busio.SPI(board.SCK, MOSI=board.MOSI)
chip_select_pin = board.D25
framebuffer = sharpdisplay.SharpMemoryFramebuffer(bus, chip_select_pin, width=144, height=168)
display = framebufferio.FramebufferDisplay(framebuffer)
display.root_group = displayio.CIRCUITPYTHON_TERMINAL
while True:
    print("hello")
    pass

The Waveshare round 1.28 LCD is a set of cheap devices which run CircuitPython and have a USB connection. They have a lot of power in a tiny formfactor.

Code

• my module waveshare128.py to expose all waveshare specific functions
• my example code to show off using touch and other functions
• another device repo for the waveshare

Boot

Hold down the boot button on the back of the device while plugging in the USB-C cable to your laptop.

Display

To access the display you need to install gc9a01, a separate driver library, with circup install gc9a01 then initialize it. Note that the touch and non-touch versions are slightly different. On the non-touch version reset is set to LCD_RST. On the touch version it is Pin 13 (board.GP13), so initialize it like this:

spi = busio.SPI(clock=board.LCD_CLK, MOSI=board.LCD_DIN)
# LCD_RST is 12 in regular version
# but we need 13 for the touch version
display_bus = displayio.FourWire(spi, 
   command=board.LCD_DC, 
   chip_select=board.LCD_CS,
   reset=board.GP13)
display = gc9a01.GC9A01(display_bus, 
   width=240, 
   height=240, 
   backlight_pin=board.LCD_BL)

The T-Deck firmware can be downloaded from the usual location but it cannot enter bootloader mode as a USB drive like most other devices. Instead you must install CircuitPython using WebSerial, which only works in Chrome.

To install it, go to the CircuitPython download page in Chrome and click the Open Installer button. A dialog will appear. Select 'Upgrade CircuitPython 9.2.1 Bin Only', choose the device in the Chrome web-serial port selection dialog, and follow the prompts. This will download and reflash the device. Then turn the device on and off with the power switch and it should boot up with the usual CIRCUITPY drive.

Under Project Settings / Python interpreter click the + button to install new packages. Install circuitpython-stubs, and then the libraries you are using. common ones include:

• adafruit-circuitpython-connectionmanager
• adafruit-circuitpython-requests
• adafruit-circuitpython-ntp

Full details here.

Install Circup

pip install setuptools
pip install circup
circup --version

Commands

• List all out of date modules: circup list
• update all out of date modules: circup update
• install a new module: circup install <module_name>
• show circup show

Circup failure

If the latest build of CP is bad for some reason, Circup can fail because it will always try to download the latest release. There should be an option to not check for the latest release. If you are installing a lib then you probably want the version of that lib that matches the CP on your board. Something like this:

circup --use-current-version install adafruit_bitmapsaver

pip install setuptools
pip install circup
circup --version

Python on Mac is a pain because there are so many different ways to do it. I've found that pyenv is the best way to manage python versions on MacOS

# install pyenv
brew install pyenv pyenv-virtualenv

# list all available python variants
pyenv install -l

# install latest 3.x.x python
pyenv install 3 

# add to the shell
echo 'PATH=$(pyenv root)/shims:$PATH' >> ~/.zshrc

Now restart your shell / open a new terminal window.

You can manage python with these commands:

• See the current global version: pyenv versions
• See where the current python versions come from: pyenv versions
• See what is the current local version: pyenv local
• List all possible versions of python to install pyenv install --list
• Install (download and build) a particular version of python pyenv install 3.12.8
• Set the current local version to 3.12.8: pyenv local 3.12.8

To convert between audio files first install ffmpeg with brew install ffmpeg.

Convert wav file to mp3. the 320k is the bitrate. For smaller files try 128k.

ffmpeg -i input.wav -codec:a libmp3lame -b:a 320k output.mp3

Sometimes you need to reformat a disk and DiskUtility isn't cutting it. I had a recent case of an SD card I'd used to install Linux. Disk Utility could see the drive but not read the linux partitions nor erase the partitions to turn it back into a plain MS-Dos (FAT) drive. The key is to use diskutil, the command line counterpart.

List the partitions

diskutil list

Find the id for the disk you are erasing. disk4 in my case.

diskutil eraseDisk FAT32 RASPBIAN MBRFormat disk4

Settings can be configured with the defaults command and the key representing that particular setting. See macOS defaults for more info.

ex:

# read the current orientation
defaults read com.apple.dock "orientation"
# position the dock on the left side of the screen
defaults write com.apple.dock "orientation" -string "left"
# restart the Dock
killall Dock 

set the default location for screenshots

# set to ~/Desktop
defaults write com.apple.screencapture "location" -string "~/Documents/Screenshots"
# restart system ui server
killall SystemUIServer

Suppose you are developing a web app on your laptop and you want your phone to test it. Getting your phone to talk to the local webserver can be tricky depending on the local network environment. Instead use an SSH tunnel to expose a single port of your laptop as a port on a remote webserver. This is called an SSH tunnel. Assuming you already have ssh locally and a linux remote webserver.

Check the /etc/ssh/sshd_config file on your server has these turned on

AllowTcpForwarding yes
GatewayPorts yes

and restart the ssh server if necessary with sudo systemctl restart ssh.

Then on your local computer set up the tunnel with

ssh -R <remoteport>:localhost:<localport> username@server.com

For example:

ssh -R 6000:localhost:4000 me@example.com

The local webserver is on 4000. Connect to it from http://example.com:6000/.

If you are using vite you access may be blocked for security reasons. Turn on allowed hosts in your vite config file.

  server: {
    allowedHosts:['josh.earth']
  }

brew install pearcleaner then run Pearcleaner.app.

Copy the script from here and paste into the terminal.

brew doctor
brew install nodejs

• screen capture: command-shift-5
• Choose window and mode
• Click to take the screenshot
• Control-click to put it into the clipboard instead of on disk
• Option-click to not show the window drop shadows
• Option-control-click to put into the clipboard and not show window drop shadows

echo $SHELL

Use the from syntax to generate a new color from an existing color.

button {
    --red: #ff0000;
    /* keep hue and saturation, multiply l times 1.5 */
    --light-red: hsl(from var(--red) h s calc(l*1.5));
}

The create-react-app tool is deprecated. For a similar experience I use Vite.

Build a new react + typescript project with create and follow the prompts.

npm create vite@latest react-ts my-app-name -- --template react-ts

Get callbacks when the browser goes on or offline:

const hand_online = () => {
    console.log('we are online')
}
const hand_offline = () => {
    console.log('we are offline')
}
window.addEventListener("online", hand_online)
window.addEventListener("offline", hand_offline)

npx lets you create a command line tool that can be run without already having it installed. To turn your npm module into such a command give it should have a javascript file with a shebang line like this: #!/usr/bin/env node. This can be in your Typescript file and the compiler should make sure it ends up in the final output js file.

Then in your package.json file needs a bin entry pointing to the script like this:

{
  "bin": {
    "amx": "build/cli.js"
  }
}

Now publish your module and run it from another shell with npx -g modulename

You can run an npm script from a different directory by using the --prefix parameter

npm --prefix <path> run <command>

When you have just released a new package version and npm install in another project says the package doesn’t exist, that could be because it tried to fetch the package before it was really live on npmjs.org. The problem is that it then remembers this forever. So you need to clear out the cache with

npm cache clean --force