Information wants to be free...

Joystick Mouse on GR-SAKURA

Here is another project based on the GR-SAKURA board. It translates joystick movement into mouse movement, so it can be used on a PC. This is done reading three analog inputs from the joystick and forwarding data over an UART using the simple Microsoft Serial Mouse format.

The software is based heavily on the 30JH joystick that I happen to have. This is a 3-axis joystick, and the last axis is used to simulate the button presses.

This command in Linux can be used to get it connected:

sudo inputattach -bare /dev/ttyUSB0
          


Here's a photo of the setup:

GR-SAKURA wth Joystick


Here is the main part of the code, which implements the serial mouse protocol:

#include "uart.h"
#include "adc.h"

/* Raw ADC reference values: */
#define JOYSTICK_POS_MAX 2464
#define JOYSTICK_POS_MIN 2080 /* JOYSTICK_POS_MAX - (128 * 3) */
#define JOYSTICK_CENTER  2048
#define JOYSTICK_NEG_MIN 2016 /* JOYSTICK_NEG_MAX + (128 * 3) */
#define JOYSTICK_NEG_MAX 1632

static unsigned char prev_buttons = 0;

static unsigned char convert_adc_to_move_value(short value)
{
  /* Scaling and manual conversion to two's complement. */
  if (value > JOYSTICK_POS_MIN) {
    if (value > JOYSTICK_POS_MAX) {
      value = JOYSTICK_POS_MAX;
    }
    return (value - (JOYSTICK_POS_MIN + 1)) / 3;

  } else if (value < JOYSTICK_NEG_MIN) {
    if (value < JOYSTICK_NEG_MAX) {
      value = JOYSTICK_NEG_MAX;
    }
    return ((value - (JOYSTICK_NEG_MAX + 1)) / 3) + 128;

  } else {
    return 0; /* Filtered. */
  }
}

static unsigned char convert_adc_to_button_value(short value)
{
  if (value > JOYSTICK_POS_MIN) {
    return 0b10; /* Left button pressed. */

  } else if (value < JOYSTICK_NEG_MIN) {
    return 0b01; /* Right button pressed. */

  } else {
    return 0b00; /* No buttons pressed. */
  }
}

static short invert_adc_value(short value)
{
  if (value > JOYSTICK_CENTER) {
    return JOYSTICK_CENTER - (value - JOYSTICK_CENTER);
  } else {
    return JOYSTICK_CENTER + (JOYSTICK_CENTER - value);
  } 
}

static int mouse_format(short x_move_adc, short y_move_adc,
                        short button_adc, unsigned char *data)
{
  unsigned char x_move, y_move, buttons;

  x_move = convert_adc_to_move_value(x_move_adc);
  y_move = convert_adc_to_move_value(invert_adc_value(y_move_adc));
  buttons = convert_adc_to_button_value(button_adc);

  data[0] = 0b11000000;
  data[1] = 0b10000000;
  data[2] = 0b10000000;

  data[0] |= (buttons << 4);
  data[0] |= ((y_move >> 4) & 0b00001100);
  data[0] |= ((x_move >> 6) & 0b00000011);
  data[1] |= (x_move & 0b00111111);
  data[2] |= (y_move & 0b00111111);

  if (y_move > 0 || x_move > 0 || buttons != prev_buttons) {
    prev_buttons = buttons;
    return 1; /* Activity */
  } else {
    return 0; /* No activity */
  }
}

void mouse_loop(void)
{
  unsigned char data[3];

  while(1) {
    asm("wait");
    if (mouse_format(adc_get(0), adc_get(1), adc_get(2), data)) {
      uart0_send(data, 3);
    }
  }
}
          


Get the complete code under the MIT License here, which requires the RX GCC toolchain to compile.

Topic: Open Source, by Kjetil @ 12/12-2018, Article Link

eLua support for GR-SAKURA

I have managed to get eLua running on the GR-SAKURA board. It's an embedded version of version of The Programming Language Lua that can be run on bare-metal.

I forked the original eLua repository and made the necessary changes here on my own copy. Note however that this initial support is very basic; only limited versions of the "tmr", "uart" and "pio" modules are supported at the moment.

Here is an example Lua script that can be run on the board. Name it "autorun.lua" and put it in the "romfs/" directory to run it automatically on startup:

require "pio"
require "pd"
require "tmr"

print("Running LEDs on " .. pd.board())
print("Hold blue button to stop.")

while pio.pin.getval(7) == 1 do
  pio.pin.sethigh(0)
  tmr.delay(0,200000)
  pio.pin.sethigh(1)
  tmr.delay(0,200000)
  pio.pin.sethigh(2)
  tmr.delay(0,200000)
  pio.pin.sethigh(6)
  tmr.delay(0,200000)

  pio.pin.setlow(0)
  tmr.delay(0,200000)
  pio.pin.setlow(1)
  tmr.delay(0,200000)
  pio.pin.setlow(2)
  tmr.delay(0,200000)
  pio.pin.setlow(6)
  tmr.delay(0,200000)
end

print("Finished.")
          


Topic: Open Source, by Kjetil @ 09/11-2018, Article Link

GR-SAKURA Shell Update

I have added two new commands to the GR-SAKURA Shell. One is just a "button" command to print the state of the blue push button, known as SW2. The other "adc" command prints the binary value of the 12-bit A/D converter connected to analog inputs AN0 to AN5.

ADC command:

sakura> adc
AN0: 101111011010
AN1: 110000101110
AN2: 110001111111
AN3: 100000001000
AN4: 000000000001
AN5: 110000110001
sakura> adc
AN0: 101110111011
AN1: 110000000010
AN2: 110001010111
AN3: 110001000001
AN4: 011111000010
AN5: 000000000011
          


Button command:

sakura> button
released
sakura> button
pressed
          


Get the source code here or from my GitLab or GitHub pages.

Also consider using my updated instructions on to build the cross compilation toolchain for the Renesas RX CPU.

Topic: Open Source, by Kjetil @ 11/10-2018, Article Link

Reverse SSH Tunnel Launcher

Here is a script which helps with starting a reverse SSH tunnel. It gets the instructions from a web server, in the form of a JSON file, which determines at what time to open the tunnel(s). The JSON file also determines when to check for a JSON file next time. It operates on a daily schedule and uses the HH:MM format.

The idea is to leave this script running on a box behind a firewall. It is hardcoded to only keep the tunnel open for 5 minutes before closing it again. So to get a connection that lasts longer, a new tunnel probably has to be made manually after connecting.

The Python code:

#!/usr/bin/python

import urllib2
import json
import subprocess
import signal
import datetime
import time
import logging
import sys

config_url = "http://192.168.0.1/config.json"
ssh_host = "192.168.0.1"
ssh_port = 22
local_port = 1337
tunnel_open_time = 300 # In seconds.

tunnel = None

def alarm_handler(signum, frame):
    global tunnel
    logger.info("Terminating current tunnel.")
    tunnel.kill();
    tunnel = None

def minutes_until(minute_ts):
    if minute_ts == None:
        return (24 * 60)

    now = datetime.datetime.now().timetuple()
    now_ts = (now.tm_hour * 60) + now.tm_min

    if now_ts > minute_ts:
        # Already passed, will be next day.
        return minute_ts - now_ts + (24 * 60)
    else:
        return minute_ts - now_ts

def update_config():
    try:
        config = json.load(urllib2.urlopen(config_url))

        # Pick closest time to update next time:
        closest_config_ts = None
        for uc in config["ConfigUpdate"]:
            config_ts = (int(uc.split(":")[0]) * 60) + int(uc.split(":")[1])
            if minutes_until(config_ts) < minutes_until(closest_config_ts):
                closest_config_ts = config_ts

        tunnel_ts = list()
        for to in config["TunnelOpen"]:
            tunnel_ts.append((int(to.split(":")[0]) * 60) + int(to.split(":")[1]))

        logger.debug("Config timestamp: %d" % (closest_config_ts))
        logger.debug("Tunnel timestamps: %s" % (str(tunnel_ts)))

        return closest_config_ts, tunnel_ts

    except Exception, e:
        logger.error("Exception during config update: %s" % (e))
        return 0, []

if __name__ == "__main__":
    logger = logging.getLogger('reverse_tunnel')
    logger.setLevel(logging.DEBUG)
    logger.addHandler(logging.StreamHandler())

    # Get initial configuration:
    config_ts, tunnel_ts = update_config()
    if len(tunnel_ts) == 0:
        sys.exit(1) # Exit right away if initial configuration fails.

    signal.signal(signal.SIGALRM, alarm_handler)

    while True:
        now = datetime.datetime.now().timetuple()
        now_ts = (now.tm_hour * 60) + now.tm_min

        # Is it time for a config update?
        if now_ts == config_ts:
            logger.info("Timed config update.")
            config_ts, tunnel_ts = update_config()
            if config_ts == now_ts:
                config_ts = config_ts - 2 # Prevent immediate update.

        # Is it time to open a new tunnel?
        for ts in tunnel_ts:
            if now_ts == ts:
                if tunnel == None:
                    logger.info("Opening new tunnel.")
                    tunnel = subprocess.Popen(["ssh", "-R", str(local_port) + ":localhost:22", "-N", "-p", str(ssh_port), ssh_host])
                    signal.alarm(tunnel_open_time)
                else:
                    logger.warning("Tunnel already running.")

        time.sleep(10)
          


Example JSON file:

{
  "ConfigUpdate" : ["17:00"],
  "TunnelOpen" : ["18:00","19:00","20:00"]
}
          


Topic: Scripts and Code, by Kjetil @ 29/09-2018, Article Link

DOS Serial File Transfer

I was doing research on how to "bootstrap" an old DOS based computer with more functional software, like a proper terminal emulator. Because the floppy drive was broken, I had to get this transferred over the existing RS-232 serial port.

The initial results were a couple of programs: A "decode" program meant to be run on DOS, written in x86 assembly, to receive the data. An "encode" program meant to be run on Linux, written in C, to send the data. Very simple encoding is used; each byte is split into two nibbles and added with 0x30 to produce a valid ASCII character.

Unfortunately I was not able to run the programs with larger amounts of data without loosing bits and pieces. Even at very low baud rates like 1200 data still got lost and enabling/disabling flow control made no difference. However, I found that other people have been suffering from these kinds of issues in the past as well. So I found some other alternatives, and the best one so far is Kermit. The MS-DOS Kermit package also provides a BASIC bootstrap program that can be run in QBASIC on the DOS computer.

Anyway, for future reference, here is the "decode" program:
(Assemble it under Linux with NASM like so: nasm decode.asm -fbin -o decode.com)

org 0x100 ; DOS COM file start offset.

section .text
start:
  ; Call BIOS to initialize serial port.
  mov ah, 0x00
  mov al, 0xe3 ; 9600,8,n,1
  mov dx, 0x00 ; COM1
  int 0x14

  ; Call DOS to create new file and handle.
  mov ah, 0x3c
  mov cx, 0 ; Standard attributes.
  mov dx, filename
  int 0x21
  jc end_file_create_error
  mov [filehandle], ax

read_loop:
  ; Call BIOS to read byte from serial port.
  mov ah, 0x02
  mov dx, 0x00 ; COM1
  int 0x14
  and ah, 0x80
  jnz end_read_serial_error

  ; Check if DOS-style EOF.
  cmp al, 0x1A
  je end_success

  ; Determine high or low nibble next.
  mov bl, [have_seen_high_nibble]
  cmp bl, 0
  jne convert_low_nibble

convert_high_nibble:
  sub al, 0x30
  mov cl, 4
  shl al, cl
  mov [high_nibble], al
  mov byte [have_seen_high_nibble], 1
  jmp read_loop

convert_low_nibble:
  sub al, 0x30
  or al, [high_nibble]
  mov byte [have_seen_high_nibble], 0

  ; Call DOS to write to file.
  mov [write_buffer], al
  mov ah, 0x40
  mov bx, [filehandle]
  mov cx, 1 ; One byte at a time.
  mov dx, write_buffer
  int 0x21
  jc end_file_write_error

  ; Call DOS to display '.' for progress.
  mov ah, 0x2
  mov dl, '.'
  int 0x21

  jmp read_loop

end_success:
  ; Call DOS to close file handle.
  mov ah, 0x3e
  mov bx, [filehandle]
  int 0x21

  ; Call DOS to terminate program.
  mov ah, 0x4c
  mov al, 0 ; 0 = OK
  int 0x21

end_file_create_error:
  mov [error_code], ax
  ; Call DOS to display 'C' signifying "Create Error".
  mov ah, 0x2
  mov dl, 'C'
  int 0x21

  ; Call DOS to display error code.
  ; mov ah, 0x2
  mov dl, [error_code]
  add dl, 0x30
  int 0x21
  jmp end_error

end_read_serial_error:
  ; Call DOS to display 'R' signifying "Read Error".
  mov ah, 0x2
  mov dl, 'R'
  int 0x21

  ; Call BIOS to get serial port status
  mov ah, 03
  mov dx, 0x00 ; COM1
  int 0x14
  mov [error_code], ax

  ; Call DOS to display error code(s).
  mov ah, 0x2
  mov dl, [error_code] ; Modem Status
  mov cl, 4
  shr dl, cl ; High Nibble
  add dl, 0x30
  int 0x21

  ; mov ah, 0x2
  mov dl, [error_code] ; Modem Status
  and dl, 0x0f ; Low Nibble
  add dl, 0x30
  int 0x21

  ; mov ah, 0x2
  mov dl, [error_code+1] ; Port Status
  mov cl, 4
  shr dl, cl ; High Nibble
  add dl, 0x30
  int 0x21

  ; mov ah, 0x2
  mov dl, [error_code+1] ; Port Status
  and dl, 0x0f ; Low Nibble
  add dl, 0x30
  int 0x21
  jmp end_error

end_file_write_error:
  mov [error_code], ax
  ; Call DOS to display 'W' signifying "Write Error".
  mov ah, 0x2
  mov dl, 'W'
  int 0x21

  ; Call DOS to display error code.
  ; mov ah, 0x2
  mov dl, [error_code]
  add dl, 0x30
  int 0x21

end_error:
  ; Call DOS to terminate program.
  mov ah, 0x4c
  mov al, 1 ; 1 = Error
  int 0x21
 
section .data:

high_nibble:
  db 0

have_seen_high_nibble:
  db 0

error_code:
filehandle:
  dw 0

write_buffer:
  db 0

filename:
  db "DECODE.BIN",0
          


And the "encode" program:

#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <termios.h>
#include <sys/ioctl.h>

int main(int argc, char *argv[])
{
  int c, fd, result;
  unsigned char byte;
  struct termios attr;

  if (argc != 2) {
    fprintf(stderr, "Usage: %s <TTY>\n", argv[0]);
    return 1;
  }

  fd = open(argv[1], O_RDWR | O_NOCTTY);
  if (fd == -1) {
    fprintf(stderr, "open() failed with errno: %d\n", errno);
    return 1;
  }

  result = tcgetattr(fd, &attr);
  if (result == -1) {
    fprintf(stderr, "tcgetattr() failed with errno: %d\n", errno);
    close(fd);
    return 1;
  }

  attr.c_cflag = B9600 | CS8 | CRTSCTS | CLOCAL;
  attr.c_iflag = 0;
  attr.c_oflag = 0;
  attr.c_lflag = 0;

  result = tcsetattr(fd, TCSANOW, &attr);
  if (result == -1) {
    fprintf(stderr, "tcgetattr() failed with errno: %d\n", errno);
    close(fd);
    return 1;
  }

  while ((c = fgetc(stdin)) != EOF) {
    byte = ((c & 0xf0) >> 4) + 0x30;
    write(fd, &byte, 1);
    byte = (c & 0x0f) + 0x30;
    write(fd, &byte, 1);
  }

  byte = 0x1a; /* DOS EOF */
  write(fd, &byte, 1);

  close(fd);
  return 0;
}
          


Topic: Scripts and Code, by Kjetil @ 19/08-2018, Article Link

File Batch Splitter

This is a Python script that takes a directory with a lot of files and splits the files among subfolder batches of a certain size. I've hard-coded the size of a standard CD (~700MB) into the script since this is a very typical use case.

Have a look:

#!/usr/bin/python

import os

class Batch(object):
    def __init__(self):
        self._size = 0
        self._files = list()

    def size_get(self):
        return self._size

    def files_get(self):
        return self._files

    def add(self, path):
        self._files.append(path)
        self._size += os.stat(path).st_size

class BatchManager(object):
    def __init__(self, max_size):
        self._max_size = max_size
        self._batches = list()

    def create(self, directory):
        batch = Batch()
        for filename in sorted(os.listdir(directory)):
            path = os.path.join(directory, filename)
            if os.path.isfile(path):
                if (batch.size_get() + os.stat(path).st_size) > self._max_size:
                    self._batches.append(batch)
                    batch = Batch()
                batch.add(path)
        self._batches.append(batch)

    def split(self, directory):
        for batch_no, batch in enumerate(self._batches):
            print "\nBatch #%03d, Size: %d" % (batch_no + 1, batch.size_get())
            for file_no, src_path in enumerate(batch.files_get()):
                print "%03d:%03d: %s" % (batch_no + 1, file_no + 1, src_path)
                batch_dir = "%03d" % (batch_no + 1)
                dst_dir = os.path.join(os.path.dirname(src_path), batch_dir)
                dst_path = os.path.join(os.path.dirname(src_path), batch_dir, os.path.basename(src_path))
                if not os.path.isdir(dst_dir):
                    os.mkdir(dst_dir)
                os.rename(src_path, dst_path)

if __name__ == "__main__":
    import sys
    if len(sys.argv) < 2:
        print "Usage: %s <directory>" % (sys.argv[0])
        sys.exit(1)

    directory = sys.argv[1]

    bm = BatchManager(737280000) # Bytes on 80 min CD-ROM Mode 1
    bm.create(directory)
    bm.split(directory)

    sys.exit(0)
          


Topic: Scripts and Code, by Kjetil @ 16/07-2018, Article Link

VPN Through SSH Tunnel

Not just VPN through an SSH tunnel, but also a Wi-Fi hotspot that directs all traffic through it! I needed this to be able to reach the Google Play store on my Android phone in China. I used two WLAN interfaces for this, but it should be possible with one WLAN and one wired connection as well.

VPN through SSH tunnel principle diagram.


Here are the complete steps, with the contents of the files listed after. Many of the commands are started through screen sessions, since they are daemons that will keep running.

First of all, make sure the interface that will be used to connect to the remote SSH server is up and running, then connect to the server and create the SSH tunnel for port 1194, which is used for VPN:

ssh user@your-ssh-server.com -L 127.0.0.1:1194:127.0.0.1:1194
          


One the remote server, enter the following commands:

# Start OpenVPN server:
screen -S openvpn -d -m sudo openvpn server.cfg

# Forward all the traffic from the OpenVPN tunnel interface:
sudo iptables -A INPUT -i tun0 -j ACCEPT
sudo iptables -A FORWARD -i tun0 -j ACCEPT
sudo iptables -A FORWARD -i eth0 -d 10.8.0.0/255.255.255.0 -j ACCEPT

# Enable IP forwarding on server:
sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
          


Back on the local machine, enter the following commands:

# IP address for Wi-Fi hotspot:
sudo ifconfig wlan0 192.168.8.1 netmask 255.255.255.0

# Start DHCP server for Wi-Fi hotspot:
screen -S dnsmasq -d -m sudo dnsmasq --conf-file=dnsmasq.conf --no-daemon

# Start Host-AP deamon:
screen -S hostapd -d -m sudo hostapd hostapd.conf

# Start OpenVPN client:
screen -S openvpn -d -m sudo openvpn client.cfg

# Let the SSH connection bypass the VPN default route:
SSH_IF='wlan1'
SSH_IP=`host your-ssh-server.com | sed -e 's/.*address //'`
GW_IP=`route -n | grep "^0.0.0.0" | grep $SSH_IF | sed -e 's/0.0.0.0 *//' | sed -e 's/ .*//'`
sudo route add $SSH_IP gw $GW_IP $SSH_IF

# Override the DNS resolver:
sudo cp resolv.conf /etc/resolv.conf

# Enable IP forwarding on local machine:
sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

# Enable NATing through the tunnel interface from the Wi-Fi hotspot:
sudo iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE
sudo iptables -A FORWARD -i tun0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A FORWARD -i wlan0 -o tun0 -j ACCEPT
          


That should be it, now the phone can connect to the local Wi-Fi hotspot and reach the Internet through the remote SSH server.

Some useful commands for troubleshooting:

sudo iptables -L -v -n
sudo tcpdump -i tun0
          


OpenVPN client configuration. (client.cfg):

remote 127.0.0.1
proto tcp-client
dev tun
ifconfig 10.8.0.2 10.8.0.1
secret static.key
redirect-gateway local def1
route 192.168.0.0 255.255.255.0
          


OpenVPN server configuration. (server.cfg):

dev tun
local 127.0.0.1
proto tcp-server
ifconfig 10.8.0.1 10.8.0.2
secret static.key
          


New DNS resolver configuration, using the Google DNS. This is important because it could have been set by a local DHCP client to a Chinese DNS resolver. (resolv.conf):

nameserver 8.8.8.8
          


DHCP server configuration. (dnsmasq.conf):

interface=lo,wlan0
no-dhcp-interface=lo
dhcp-range=192.168.8.20,192.168.8.254,255.255.255.0,12h
          


Host-AP Wi-Fi hotspot configuration. These are the more important options, I have excluded lots of others. (hostapd.conf):

interface=wlan0
driver=nl80211
ssid=YourSSID
hw_mode=g
wpa=2
wpa_passphrase=YourPassphrase
wpa_key_mgmt=WPA-PSK WPA-EAP
          


Topic: Configuration, by Kjetil @ 10/06-2018, Article Link

Python Image Viewer

Here is Python script I quickly made for browsing through images, and with a button to pass on the filepath of the currently viewed image to another command. However, it's very limited and doesn't even resize images, maybe that and more will be fixed in a future revision.

The code:

#!/usr/bin/python

from Tkinter import *
from PIL import ImageTk, Image
import subprocess
import os.path

class ImageView(Frame):
    def __init__(self, images, command, master=None):
        Frame.__init__(self, master)
        self.grid()
        self._create_widgets()
        self._images = images
        self._command = command
        self._offset = 0
        self._browse(0) # Load first image!

    def _browse(self, offset):
        self._offset += offset
        if self._offset < 0:
            self._offset = 0
        elif self._offset >= len(self._images):
            self._offset = (len(self._images) - 1)

        self.master.title("%s (%d/%d)" % (os.path.basename(self._images[self._offset]), self._offset + 1, len(self._images)))

        self._image_data = ImageTk.PhotoImage(Image.open(self._images[self._offset]))
        self._image = Button(image=self._image_data, bd=0, relief=FLAT, command=lambda: self._browse(1))
        self._image.grid(column=0, row=0, columnspan=7, sticky="news")

    def _exec(self):
        subprocess.call([self._command, self._images[self._offset]])

    def _create_widgets(self):
        self._prev_1   = Button(text="<",   command=lambda: self._browse(-1))
        self._prev_10  = Button(text="<<",  command=lambda: self._browse(-10))
        self._prev_100 = Button(text="<<<", command=lambda: self._browse(-100))
        self._next_1   = Button(text=">",   command=lambda: self._browse(1))
        self._next_10  = Button(text=">>",  command=lambda: self._browse(10))
        self._next_100 = Button(text=">>>", command=lambda: self._browse(100))
        self._exec     = Button(text="Go!", command=self._exec)

        self._prev_100.grid(column=0, row=1, sticky="ew")
        self._prev_10.grid (column=1, row=1, sticky="ew")
        self._prev_1.grid  (column=2, row=1, sticky="ew")
        self._exec.grid    (column=3, row=1, sticky="ew")
        self._next_1.grid  (column=4, row=1, sticky="ew")
        self._next_10.grid (column=5, row=1, sticky="ew")
        self._next_100.grid(column=6, row=1, sticky="ew")

def image_walker(arg, dirname, names):
    for fname in names:
        path = os.path.join(dirname, fname)
        if path.endswith(".jpg"):
            arg.append(path)

if __name__ == "__main__":
    import sys

    if len(sys.argv) < 3:
        print "Usage: %s <image directory> <command>" % (sys.argv[0])
        sys.exit(1)

    images = list()
    os.path.walk(sys.argv[1], image_walker, images)
    if len(images) == 0:
        print "No images found :-("
        sys.exit(1)

    iw = ImageView(sorted(images), sys.argv[2])
    iw.master.resizable(0, 0)
    iw.mainloop()

    sys.exit(0)
          


Topic: Scripts and Code, by Kjetil @ 21/05-2018, Article Link

USB-CAN Analyzer Linux Support

I already put this code on my GitHub page earlier, but I'm posting it on my page as well for completeness sake.

This is a small C program that dumps the CAN traffic for one the USB-CAN adapters found everywhere on Ebay nowadays. The manufacturer does not support Linux by themselves, and only gives the link to a Windows binary program.

When plugged in, it will show something like this:

Bus 002 Device 006: ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter
          


So the whole thing is actually a USB to serial converter, for which Linux will provide the 'ch341-uart' driver and create a new /dev/ttyUSB device. This program simply implements part of that serial protocol.

The code:

#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <asm/termbits.h> /* struct termios2 */
#include <time.h>

#define CANUSB_BAUD_RATE 2000000

typedef enum {
  CANUSB_SPEED_1000000 = 0x01,
  CANUSB_SPEED_800000  = 0x02,
  CANUSB_SPEED_500000  = 0x03,
  CANUSB_SPEED_400000  = 0x04,
  CANUSB_SPEED_250000  = 0x05,
  CANUSB_SPEED_200000  = 0x06,
  CANUSB_SPEED_125000  = 0x07,
  CANUSB_SPEED_100000  = 0x08,
  CANUSB_SPEED_50000   = 0x09,
  CANUSB_SPEED_20000   = 0x0a,
  CANUSB_SPEED_10000   = 0x0b,
  CANUSB_SPEED_5000    = 0x0c,
} CANUSB_SPEED;

typedef enum {
  CANUSB_MODE_NORMAL          = 0x00,
  CANUSB_MODE_LOOPBACK        = 0x01,
  CANUSB_MODE_SILENT          = 0x02,
  CANUSB_MODE_LOOPBACK_SILENT = 0x03,
} CANUSB_MODE;

typedef enum {
  CANUSB_FRAME_STANDARD = 0x01,
  CANUSB_FRAME_EXTENDED = 0x02,
} CANUSB_FRAME;

static int print_traffic = 0;

static CANUSB_SPEED canusb_int_to_speed(int speed)
{
  switch (speed) {
  case 1000000:
    return CANUSB_SPEED_1000000;
  case 800000:
    return CANUSB_SPEED_800000;
  case 500000:
    return CANUSB_SPEED_500000;
  case 400000:
    return CANUSB_SPEED_400000;
  case 250000:
    return CANUSB_SPEED_250000;
  case 200000:
    return CANUSB_SPEED_200000;
  case 125000:
    return CANUSB_SPEED_125000;
  case 100000:
    return CANUSB_SPEED_100000;
  case 50000:
    return CANUSB_SPEED_50000;
  case 20000:
    return CANUSB_SPEED_20000;
  case 10000:
    return CANUSB_SPEED_10000;
  case 5000:
    return CANUSB_SPEED_5000;
  default:
    return 0;
  }
}

static int generate_checksum(unsigned char *data, int data_len)
{
  int i, checksum;

  checksum = 0;
  for (i = 0; i < data_len; i++) {
    checksum += data[i];
  }

  return checksum & 0xff;
}

static int frame_is_complete(unsigned char *frame, int frame_len)
{
  if (frame_len > 0) {
    if (frame[0] != 0xaa) {
      /* Need to sync on 0xaa at start of frames, so just skip. */
      return 1;
    }
  }

  if (frame_len < 2) {
    return 0;
  }

  if (frame[1] == 0x55) { /* Command frame... */
    if (frame_len >= 20) { /* ...always 20 bytes. */
      return 1;
    } else {
      return 0;
    }
  } else if ((frame[1] >> 4) == 0xc) { /* Data frame... */
    if (frame_len >= (frame[1] & 0xf) + 5) { /* ...payload and 5 bytes. */
      return 1;
    } else {
      return 0;
    }
  }

  /* Unhandled frame type. */
  return 1;
}

static int frame_send(int tty_fd, unsigned char *frame, int frame_len)
{
  int result, i;

  if (print_traffic) {
    printf(">>> ");
    for (i = 0; i < frame_len; i++) {
      printf("%02x ", frame[i]);
    }
    printf("\n");
  }

  result = write(tty_fd, frame, frame_len);
  if (result == -1) {
    fprintf(stderr, "write() failed: %s\n", strerror(errno));
    return -1;
  }

  return frame_len;
}

static int frame_recv(int tty_fd, unsigned char *frame, int frame_len_max)
{
  int result, frame_len, checksum;
  unsigned char byte;

  if (print_traffic)
    fprintf(stderr, "<<< ");

  frame_len = 0;
  while (1) {
    result = read(tty_fd, &byte, 1);
    if (result == -1) {
      if (errno != EAGAIN && errno != EWOULDBLOCK) {
        fprintf(stderr, "read() failed: %s\n", strerror(errno));
        return -1;
      }

    } else if (result > 0) {
      if (print_traffic)
        fprintf(stderr, "%02x ", byte);

      if (frame_len == frame_len_max) {
        fprintf(stderr, "frame_recv() failed: Overflow\n");
        return -1;
      }

      frame[frame_len++] = byte;

      if (frame_is_complete(frame, frame_len)) {
        break;
      }
    }

    usleep(10);
  }

  if (print_traffic)
    fprintf(stderr, "\n");

  /* Compare checksum for command frames only. */
  if ((frame_len == 20) && (frame[0] == 0xaa) && (frame[1] == 0x55)) {
    checksum = generate_checksum(&frame[2], 17);
    if (checksum != frame[frame_len - 1]) {
      fprintf(stderr, "frame_recv() failed: Checksum incorrect\n");
      return -1;
    }
  }

  return frame_len;
}

static int command_settings(int tty_fd, CANUSB_SPEED speed, CANUSB_MODE mode, CANUSB_FRAME frame)
{
  int cmd_frame_len;
  unsigned char cmd_frame[20];

  cmd_frame_len = 0;
  cmd_frame[cmd_frame_len++] = 0xaa;
  cmd_frame[cmd_frame_len++] = 0x55;
  cmd_frame[cmd_frame_len++] = 0x12;
  cmd_frame[cmd_frame_len++] = speed;
  cmd_frame[cmd_frame_len++] = frame;
  cmd_frame[cmd_frame_len++] = 0; /* Filter ID not handled. */
  cmd_frame[cmd_frame_len++] = 0; /* Filter ID not handled. */
  cmd_frame[cmd_frame_len++] = 0; /* Filter ID not handled. */
  cmd_frame[cmd_frame_len++] = 0; /* Filter ID not handled. */
  cmd_frame[cmd_frame_len++] = 0; /* Mask ID not handled. */
  cmd_frame[cmd_frame_len++] = 0; /* Mask ID not handled. */
  cmd_frame[cmd_frame_len++] = 0; /* Mask ID not handled. */
  cmd_frame[cmd_frame_len++] = 0; /* Mask ID not handled. */
  cmd_frame[cmd_frame_len++] = mode;
  cmd_frame[cmd_frame_len++] = 0x01;
  cmd_frame[cmd_frame_len++] = 0;
  cmd_frame[cmd_frame_len++] = 0;
  cmd_frame[cmd_frame_len++] = 0;
  cmd_frame[cmd_frame_len++] = 0;
  cmd_frame[cmd_frame_len++] = generate_checksum(&cmd_frame[2], 17);

  if (frame_send(tty_fd, cmd_frame, cmd_frame_len) < 0) {
    return -1;
  }

  return 0;
}

static void dump_data_frames(int tty_fd)
{
  int i, frame_len;
  unsigned char frame[32];
  struct timespec ts;

  while (1) {
    frame_len = frame_recv(tty_fd, frame, sizeof(frame));

    clock_gettime(CLOCK_MONOTONIC, &ts);
    printf("%lu.%06lu ", ts.tv_sec, ts.tv_nsec / 1000);

    if (frame_len == -1) {
      printf("Frame recieve error!\n");

    } else {

      if ((frame_len >= 6) &&
          (frame[0] == 0xaa) &&
          ((frame[1] >> 4) == 0xc)) {
        printf("Frame ID: %02x%02x, Data: ", frame[3], frame[2]);
        for (i = frame_len - 2; i > 3; i--) {
          printf("%02x ", frame[i]);
        }
        printf("\n");

      } else {
        printf("Unknown:");
        for (i = 0; i <= frame_len; i++) {
          printf("%02x ", frame[i]);
        }
        printf("\n");
      }
    }
  }
}

static int adapter_init(char *tty_device)
{
  int tty_fd, result;
  struct termios2 tio;

  tty_fd = open(tty_device, O_RDWR | O_NOCTTY | O_NONBLOCK);
  if (tty_fd == -1) {
    fprintf(stderr, "open(%s) failed: %s\n", tty_device, strerror(errno));
    return -1;
  }
 
  result = ioctl(tty_fd, TCGETS2, &tio);
  if (result == -1) {
    fprintf(stderr, "ioctl() failed: %s\n", strerror(errno));
    close(tty_fd);
    return -1;
  }

  tio.c_cflag &= ~CBAUD;
  tio.c_cflag = BOTHER | CS8 | CSTOPB;
  tio.c_iflag = IGNPAR;
  tio.c_oflag = 0;
  tio.c_lflag = 0;
  tio.c_ispeed = CANUSB_BAUD_RATE;
  tio.c_ospeed = CANUSB_BAUD_RATE;

  result = ioctl(tty_fd, TCSETS2, &tio);
  if (result == -1) {
    fprintf(stderr, "ioctl() failed: %s\n", strerror(errno));
    close(tty_fd);
    return -1;
  }

  return tty_fd;
}

static void display_help(char *progname)
{
  fprintf(stderr, "Usage: %s <options>\n", progname);
  fprintf(stderr, "Options:\n"
     "  -h          Display this help and exit.\n"
     "  -t          Print TTY/serial traffic debugging info on stderr.\n"
     "  -d DEVICE   Use TTY DEVICE.\n"
     "  -s SPEED    Set CAN SPEED in bps.\n"
     "\n");
}

int main(int argc, char *argv[])
{
  int c, tty_fd;
  char *tty_device = NULL;
  CANUSB_SPEED speed = 0;

  while ((c = getopt(argc, argv, "htd:s:")) != -1) {
    switch (c) {
    case 'h':
      display_help(argv[0]);
      return EXIT_SUCCESS;

    case 't':
      print_traffic = 1;
      break;

    case 'd':
      tty_device = optarg;
      break;

    case 's':
      speed = canusb_int_to_speed(atoi(optarg));
      break;

    case '?':
    default:
      display_help(argv[0]);
      return EXIT_FAILURE;
    }
  }

  if (tty_device == NULL) {
    fprintf(stderr, "Please specify a TTY!\n");
    display_help(argv[0]);
    return EXIT_FAILURE;
  }

  if (speed == 0) {
    fprintf(stderr, "Please specify a valid speed!\n");
    display_help(argv[0]);
    return EXIT_FAILURE;
  }

  tty_fd = adapter_init(tty_device);
  if (tty_fd == -1) {
    return EXIT_FAILURE;
  }

  command_settings(tty_fd, speed, CANUSB_MODE_NORMAL, CANUSB_FRAME_STANDARD);
  dump_data_frames(tty_fd);

  return EXIT_SUCCESS;
}
          


Topic: Scripts and Code, by Kjetil @ 02/04-2018, Article Link

Renesas GCC Toolchains

The gcc-renesas.com webpage has good information on how to build the Renesas RX and RL78 toolchains, but the GCC versions used there are a little outdated. And in addition, some patches are applied.

So instead I tried to build the toolchains from scratch directly from the GNU sources, and also get the benefit of the newer GCC version 7 instead of GCC version 4.

My efforts were successful and I have automated it into a single script. Simply change the TARGET variable to build either RL78 or the RX toolchain. It installs everything into a separate directoy in /opt/ on the filesystem. Take a look:

#!/bin/bash
set -e

TARGET="rl78-elf"
#TARGET="rx-elf"
PREFIX="/opt/gcc-${TARGET}/"

export PATH="${PREFIX}bin:$PATH"

# 1) Prepare build directories:
if [ -d build ]; then
  echo "Old build directory detected, please remove it."
  exit 1
else
  mkdir -p build/autoconf
  mkdir -p build/binutils
  mkdir -p build/gcc
  mkdir -p build/gdb
  mkdir -p build/newlib
fi

# 2) Get sources:
if [ ! -d source ]; then
  mkdir source
  cd source
  wget "https://gnuftp.uib.no/autoconf/autoconf-2.64.tar.bz2"
  wget "https://gnuftp.uib.no/gcc/gcc-7.3.0/gcc-7.3.0.tar.xz"
  wget "https://gnuftp.uib.no/gdb/gdb-8.1.tar.xz"
  wget "https://gnuftp.uib.no/binutils/binutils-2.30.tar.xz"
  wget "ftp://sourceware.org/pub/newlib/newlib-2.5.0.tar.gz"
  tar -xvjf autoconf-2.64.tar.bz2
  tar -xvJf gcc-7.3.0.tar.xz
  tar -xvJf gdb-8.1.tar.xz
  tar -xvJf binutils-2.30.tar.xz
  tar -xvzf newlib-2.5.0.tar.gz
  cd ..
fi

# 3) Build autoconf:
cd build/autoconf
../../source/autoconf-2.64/configure --prefix=$PREFIX
make
sudo make install
cd ..

# 4) Build binutils:
cd binutils
../../source/binutils-2.30/configure --target=$TARGET --prefix=$PREFIX --enable-maintainer-mode --disable-nls --disable-werror
make
sudo make install
cd ..

# 5) Get gcc sources:
if [ ! -d ../source/gcc-7.3.0/gmp ]; then
  cd ../source/gcc-7.3.0
  ./contrib/download_prerequisites
  cd ../../build
fi

# 6) Build gcc (step 1):
cd gcc
../../source/gcc-7.3.0/configure --target=$TARGET --prefix=$PREFIX --enable-languages=c,c++ --disable-shared --with-newlib --enable-lto --disable-libstdcxx-pch --disable-nls --disable-werror
make all-gcc
sudo make install-gcc
cd ..

# 7) Build newlib:
cd newlib
../../source/newlib-2.5.0/configure --target=$TARGET --prefix=$PREFIX --disable-nls
make
sudo make install
cd ..

# 8) Build gdb:
cd gdb
../../source/gdb-8.1/configure --target=$TARGET --prefix=$PREFIX --disable-nls
make
sudo make install
cd ..

# 9) Build gcc (step 2):
cd gcc
make
sudo make install
          


Topic: Configuration, by Kjetil @ 03/03-2018, Article Link

GR-SAKURA Shell

Renesas has more microcontroller reference boards. On another trip to Japan I picked up the GR-SAKURA board, which uses the Renesas RX63N microcontroller.

Like the GR-KURUMI, I have made a command shell interface for the GR-SAKURA as well, based on much of the same code, to control its LED and timer functions.

The shell is spawned as the UART on "SCI0", which has the RxD pin at P21 (aka IO0) and the TxD pin at P20 (aka IO1) on the board. The code and build environment also assumes that the pre-flashed USB mass-storage based bootloader is in use, so the binary may be transferred using that USB interface.

Get the source code here and follow this advice on how to setup the toolchain for compiling Renesas RX binaries. I have created custom C-runtime stubs and a linker script to work specifically with the pre-flashed bootloader. These are included in the source code tarball as well.

Here is a similar Python script that can be used to load script files remotely:

#!/usr/bin/python

import serial

class Sakura(object):
    def __init__(self, port):
        self.s = serial.Serial()
        self.s.port = port
        self.s.baudrate = 9600
        self.s.bytesize = serial.EIGHTBITS
        self.s.parity = serial.PARITY_NONE
        self.s.stopbits = serial.STOPBITS_ONE
        self.s.xonxoff = False
        self.s.rtscts = False
        self.s.dsrdtr = False

        self.s.open()
        self.s.flushInput()
        self.s.flushOutput()
    
    def __del__(self):
        self.s.close()

    def _command(self, cmd):
        print ">", cmd
        for char in cmd:
            self.s.write(char) # Write one character at a time...
            self.s.read(1)     # ...and read the echo.
        self.s.write("\r") # Finish command...
        self.s.read(10)    # ...and read the prompt.

    def script(self, filename):
        self._command("stop")
        self._command("d1 off")
        self._command("d2 off")
        self._command("d3 off")
        self._command("d4 off")
        self._command("clear")
        with open(filename) as fh:
            for line_no, line in enumerate(fh):
                self._command("%s %s" % (line_no, line.strip()))
        self._command("run")

if __name__ == "__main__":
    import sys
    s = Sakura("/dev/ttyUSB0")

    if len(sys.argv) > 1:
        s.script(sys.argv[1])
    else:
        print "Usage: %s <script file>" % (sys.argv[0])
          


One difference compared to the GR-KURUMI is that the LEDs on GR-SAKURA are all blue and named D1 to D4 instead. Here is a sample script:

d1 on
sleep 50
d1 off
sleep 50
d2 on
d3 on
sleep 100
d2 off
sleep 100
d3 off
sleep 500
d4 toggle
          


Topic: Open Source, by Kjetil @ 04/02-2018, Article Link

GitHub Page and Curses Collection

I've had a GitHub page for quite some time, but never utilized it much. I will upload the code for some of my projects over there as well, to get a little more exposure.

As a starter, I have made a collection/compilation of most of the curses-based tools and utilities that I have created over the years. You'll find it here.

As a bonus, the collection also has a Makefile like this to build everything in one go:

PROGS = cavescroll cgame difftree fileselect invaders mastermind menu playlist scca snake snowcrash storage sudoku

.PHONY: all clean

all clean:
	for PROG in $(PROGS); do \
	  $(MAKE) -C $$PROG $@; \
	done
          


Topic: General, by Kjetil @ 11/01-2018, Article Link