Python Internet Radio Manager
In their infinite wisdom, the Norwegian government decided to shut down all FM boardcasts in favor of the even more obsolete and inferior DAB broadcast system this year. In response to their futile attempts, I have instead set up a Raspberry Pi 2 to function as an Internet radio and feed the audio into my existing radio equipment.
I only listen to the radio in the morning, so to avoid using the Internet connection at all times I needed some kind of manager. The result is a Python script which keeps track of the time of day and switches whatever is played on the audio output. This also gives the possibility to easily switch channels during the weekend.
Here is the script, where everything is just configured with my preferences within the script itself:
#!/usr/bin/python import subprocess import signal import datetime import time events = ( ("SUN", 15, 0, ["su", "root", "-c", "ntpdate 192.168.0.1"]), ("MON", 7, 45, ["mpg123", "http://nrk-mms-live.telenorcdn.net:80/nrk_radio_p1_rogaland_mp3_h"]), ("MON", 8, 30, ["./portal_looping_radio.sh"]), ("TUE", 7, 45, ["mpg123", "http://nrk-mms-live.telenorcdn.net:80/nrk_radio_p1_rogaland_mp3_h"]), ("TUE", 8, 30, ["./portal_looping_radio.sh"]), ("WED", 7, 45, ["mpg123", "http://nrk-mms-live.telenorcdn.net:80/nrk_radio_p1_rogaland_mp3_h"]), ("WED", 8, 30, ["./portal_looping_radio.sh"]), ("THU", 7, 45, ["mpg123", "http://nrk-mms-live.telenorcdn.net:80/nrk_radio_p1_rogaland_mp3_h"]), ("THU", 8, 30, ["./portal_looping_radio.sh"]), ("FRI", 7, 45, ["mpg123", "http://nrk-mms-live.telenorcdn.net:80/nrk_radio_p1_rogaland_mp3_h"]), ("FRI", 8, 30, ["./portal_looping_radio.sh"]), ("SAT", 7, 45, ["mpg123", "http://nrk-mms-live.telenorcdn.net:80/nrk_radio_jazz_mp3_h"]), ("SAT", 9, 30, ["./portal_looping_radio.sh"]), ("SUN", 7, 45, ["mpg123", "http://nrk-mms-live.telenorcdn.net:80/nrk_radio_jazz_mp3_h"]), ("SUN", 9, 30, ["./portal_looping_radio.sh"]), ) default_event = ["./portal_looping_radio.sh"] p = None def alarm_handler(signum, frame): print "[RADIO] Got Alarm!" p.kill() def event_to_secs(event): if event[0] == "MON": secs = 0 elif event[0] == "TUE": secs = 86400 elif event[0] == "WED": secs = 172800 elif event[0] == "THU": secs = 259200 elif event[0] == "FRI": secs = 345600 elif event[0] == "SAT": secs = 432000 elif event[0] == "SUN": secs = 518400 secs += (event[1] * 3600) secs += (event[2] * 60) return secs def get_next_event(): now = datetime.datetime.now().timetuple() now_secs = (now.tm_wday * 86400) + (now.tm_hour * 3600) + (now.tm_min * 60) + now.tm_sec nearest_event_secs = 604801 nearest_event = None first_event_secs = 604801 first_event = None for event in events: event_secs = event_to_secs(event) if event_secs < first_event_secs: first_event_secs = event_secs first_event = event if event_secs > now_secs and event_secs < nearest_event_secs: nearest_event_secs = event_secs nearest_event = event if nearest_event == None: return (604800 + first_event_secs) - now_secs, first_event[3] else: return nearest_event_secs - now_secs, nearest_event[3] if __name__ == "__main__": print "[RADIO] Starting" signal.signal(signal.SIGALRM, alarm_handler) p = subprocess.Popen(default_event) while True: timeleft, next_event = get_next_event() print "[RADIO] Time Left:", timeleft signal.alarm(timeleft) p.wait() print "[RADIO] Event:", next_event p = subprocess.Popen(next_event) time.sleep(30) p.poll() if p.returncode != None: print "[RADIO] Error, running default." p = subprocess.Popen(default_event) time.sleep(3) p.poll() if p.returncode != None: break print "[RADIO] Exit"
GR-KURUMI Shell
A follow-up on my previous articles about a GR-KURUMI Writer and the GR-KURUMI Makefiles. Here is a program written for the GR-KURUMI that provides a command shell interface against its LED and timer functions.
The shell is spawned on UART #0, the same one used for flashing, since this is most convenient. The DTR signal must be disconnected in order to avoid the chip going into flashing mode though. The shell provides a simple BASIC-style scripting interface. Commands can be put into a script/program buffer, indexed by 0 to 99, which can be run continuously.
Get the source code here and check the article on Makefiles on how to get the tool chain up and running. Then use the article on the Writer to flash the chip.
Here is a Python script that can be used to load script files remotely in an easy manner:
#!/usr/bin/python import serial class Kurumi(object): def __init__(self, port): self.s = serial.Serial() self.s.port = port self.s.baudrate = 9600 self.s.bytesize = serial.EIGHTBITS self.s.parity = serial.PARITY_NONE self.s.stopbits = serial.STOPBITS_ONE self.s.xonxoff = False self.s.rtscts = False self.s.dsrdtr = False self.s.open() self.s.flushInput() self.s.flushOutput() def __del__(self): self.s.close() def _command(self, cmd): print ">", cmd for char in cmd: self.s.write(char) # Write one character at a time... self.s.read(1) # ...and read the echo. self.s.write("\r") # Finish command... self.s.read(10) # ...and read the prompt. def script(self, filename): self._command("stop") self._command("red off") self._command("green off") self._command("blue off") self._command("clear") with open(filename) as fh: for line_no, line in enumerate(fh): self._command("%s %s" % (line_no, line.strip())) self._command("run") if __name__ == "__main__": import sys k = Kurumi("/dev/ttyUSB0") if len(sys.argv) > 1: k.script(sys.argv[1]) else: print "Usage: %s <script file>" % (sys.argv[0])
A typical script file can look like this:
red on sleep 50 red off sleep 50 green on blue on sleep 100 green off sleep 100 blue off sleep 500
Fluxbox Styles
I almost exclusively use the Fluxbox window manager on my systems. Recently I have looked into how to create custom styles, and have finally been able to boil it down into something that's easy to manage.
Here's a desktop screenshot of a style I made:

But the most interesting part is the trimmed down style file:
*.textColor: #009900 *.color: black *.focus.color: #009900 *.focus.textColor: black *.focused.color: #009900 *.focused.textColor: black *.font: glisp *.borderWidth: 1 *.borderColor: #009900 menu.bullet: triangle menu.bullet.position: right menu.title.color: #009900 menu.title.textColor: black menu.title.justify: center menu.frame.color: black menu.frame.textColor: green menu.hilite.color: #009900 menu.hilite.textColor: black menu.frame.disableColor: grey50 window.button.focus.picColor: black window.button.unfocus.picColor: green window.*.focus: raised gradient vertical window.*.focus.colorTo: #00ee00 toolbar.*.focused: raised gradient vertical toolbar.*.focused.colorTo: #00ee00 menu.title: raised gradient vertical menu.title.colorTo: #00ee00
In the corner of the screenshot you'll also notice conky running. Here is the conkyrc file creating a similar colorscheme to match the Fluxbox style.
alignment top_right background yes border_width 1 cpu_avg_samples 2 default_color green default_outline_color green default_shade_color green draw_borders no draw_graph_borders yes draw_outline no draw_shades no use_xft no xftfont DejaVu Sans Mono:size=12 gap_x 5 gap_y 5 minimum_size 300 5 net_avg_samples 2 no_buffers yes out_to_console no out_to_stderr no extra_newline no own_window no own_window_class Conky own_window_type desktop own_window_transparent yes stippled_borders 0 update_interval 1.0 uppercase no use_spacer none show_graph_scale no show_graph_range no double_buffer yes TEXT ${color green4}Uptime:$color $uptime ${color green4}Battery:$color $battery ${battery_bar} ${color green4}RAM Usage:$color $mem/$memmax - $memperc% ${membar 4} ${color green4}CPU \#1:$color ${cpu cpu1}% ${cpubar cpu1 4} ${color green4}CPU \#2:$color ${cpu cpu2}% ${cpubar cpu2 4} ${color green4}CPU \#3:$color ${cpu cpu3}% ${cpubar cpu3 4} ${color green4}CPU \#4:$color ${cpu cpu4}% ${cpubar cpu4 4} ${color green4}Processes:$color $processes ${color green4}Running:$color $running_processes ${color green4}Wired:$color ${addr eth0} ${color green4}Wireless:$color ${addr wlan0}
A final recommendation when using styles like this is to make sure that the "ls" command does not produce garish results. This setting helps:
export LS_COLORS='rs=00:di=01:ln=00:mh=00:pi=01:so=01:do=01:bd=01:cd=01:or=31;01:mi=00:su=00:sg=00:ca=00:tw=00:ow=00:st=00:ex=00:'
GR-KURUMI Makefiles for Linux
Here's some follow-up information on the GR-KURUMI microcontroller reference board mentioned in the previous article.
On the official Gadget Renesas pages you can find an online editor and compiler refered to as the "Web Compiler", but in case you want to build the sources on your own Linux box then some more work is required.
First of all, download and build the RL78 GCC toolchain. Follow this advice.
Second, you will need the linker script, crt0 and header files which are specific for the RL78. I got this from the Gadget Renesas Web Compiler by downloading an example project.
Finally, here is an example of a simplified Makefile, based around compiling the source file "led.c" into the binary file "led.bin", which can be flashed:
TOOL_PATH:=/path/to/RL78-Toolchain/prefix/bin CFLAGS = -c -Os -ffunction-sections -fdata-sections -I. -Icommon LDFLAGS = -Wl,--gc-sections -nostartfiles led.bin: led.elf $(TOOL_PATH)/rl78-elf-objcopy -O binary $^ $@ led.elf: led.o crt0.o $(TOOL_PATH)/rl78-elf-gcc $(LDFLAGS) -T common/rl78_R5F100GJAFB.ld $^ -o $@ crt0.o: common/crt0.S $(TOOL_PATH)/rl78-elf-gcc $(CFLAGS) $^ -o $@ led.o: led.c $(TOOL_PATH)/rl78-elf-gcc $(CFLAGS) $^ -o $@ .PHONY: clean clean: rm -f *.o *.elf *.bin
Here is the led.c example, which I have mostly just copied from the Internet and not written myself:
#include <iodefine.h> #include <iodefine_ext.h> #define LED_CYAN_PIN PM1.BIT.bit7 #define LED_MAGENTA_PIN PM5.BIT.bit1 #define LED_YELLOW_PIN PM5.BIT.bit0 #define LED_CYAN P1.BIT.bit7 #define LED_MAGENTA P5.BIT.bit1 #define LED_YELLOW P5.BIT.bit0 __attribute__((interrupt)) void wdti_handler(void) { } __attribute__((interrupt)) void it_handler(void) { LED_CYAN ^= 1; LED_MAGENTA = 0; LED_YELLOW ^= 1; } void main(void) { asm("di"); LED_CYAN_PIN = 0; LED_MAGENTA_PIN = 0; LED_YELLOW_PIN = 0; LED_CYAN = 0; LED_MAGENTA = 0; LED_YELLOW = 0; /* Setup clocks */ CMC.cmc = 0x11U; /* Enable XT1, disable X1 */ CSC.csc = 0x80U; /* Start XT1 */ CKC.ckc = 0x00U; /* Interval timer */ OSMC.osmc = 0x80U; /* Supply fsub to Interval Timer */ RTCEN = 1; ITMK = 1; /* Disable interrupt... */ ITPR0 = 0; /* High pri... */ ITPR1 = 0; ITMC.itmc = 0x8FFFU; /* 270ms... */ ITIF = 0; /* interrupt request flag... */ ITMK = 0; /* Enable interrupt... */ asm("ei"); /* Enable interrupts */ for(;;) { asm("stop"); /* STOP mode. */ } }
GR-KURUMI Writer for Linux
On a recent trip to Japan I picked up a microcontroller reference board known as GR-KURUMI, which uses the Renesas RL78/G13 microcontroller.
I was surprised to see that no flashing tool for Linux was available from the supplier. Well, I later discovered this GitHub project which can do the job. But it was already too late and I had already made my own tool by reverse engineering the serial protocol used by the Windows flashing tool.
Here are the results, the tool is made specifically for and only tested with the GR-KURUMI board:
#include <stdlib.h> #include <string.h> #include <stdio.h> #include <errno.h> #include <unistd.h> #include <sys/ioctl.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <termios.h> #define BLOCK_SIZE 1024 /* In bytes. */ typedef enum { RL78_COMMAND_RESET = 0x00, RL78_COMMAND_VERIFY = 0x13, RL78_COMMAND_BLOCK_ERASE = 0x22, RL78_COMMAND_BLOCK_BLANK_CHECK = 0x32, RL78_COMMAND_PROGRAMMING = 0x40, RL78_COMMAND_BAUD_RATE_SET = 0x9a, RL78_COMMAND_SECURITY_SET = 0xa0, RL78_COMMAND_SECURITY_GET = 0xa1, RL78_COMMAND_SECURITY_RELEASE = 0xa2, RL78_COMMAND_CHECKSUM = 0xb0, RL78_COMMAND_SILICON_SIGNATURE = 0xc0, } RL78_COMMAND; typedef enum { RL78_STATUS_COMMAND_NUMBER_ERROR = 0x04, RL78_STATUS_PARAMETER_ERROR = 0x05, RL78_STATUS_NORMAL_ACK = 0x06, RL78_STATUS_CHECKSUM_ERROR = 0x07, RL78_STATUS_VERIFY_ERROR = 0x0f, RL78_STATUS_PROTECT_ERROR = 0x10, RL78_STATUS_NEGATIVE_ACK = 0x15, RL78_STATUS_ERASE_ERROR = 0x1a, RL78_STATUS_IVERIFY_BLANK_ERROR = 0x1b, RL78_STATUS_WRITE_ERROR = 0x1c, } RL78_STATUS; static int print_traffic = 0; static int print_details = 1; static char *rl78_status_text(int status) { switch (status) { case RL78_STATUS_COMMAND_NUMBER_ERROR: return "Command number error"; case RL78_STATUS_PARAMETER_ERROR: return "Parameter error"; case RL78_STATUS_NORMAL_ACK: return "Normal acknowledgement"; case RL78_STATUS_CHECKSUM_ERROR: return "Checksum error"; case RL78_STATUS_VERIFY_ERROR: return "Verify error"; case RL78_STATUS_PROTECT_ERROR: return "Protect error"; case RL78_STATUS_NEGATIVE_ACK: return "Negative acknowledgement"; case RL78_STATUS_ERASE_ERROR: return "Erase error"; case RL78_STATUS_IVERIFY_BLANK_ERROR: return "Internal verify error or blank check error"; case RL78_STATUS_WRITE_ERROR: return "Write error"; default: return "Unknown error"; } } static int generate_checksum(unsigned char *data, int data_len) { int i, checksum; if (data_len == 0) { data_len = 0x100; } checksum = (0 - data_len); for (i = 0; i < data_len; i++) { checksum -= data[i]; } return checksum & 0xff; } static int frame_is_complete(unsigned char *frame, int frame_len) { int internal_frame_len; if (frame_len < 5) { return 0; } if (frame[1] == 0) { internal_frame_len = 0x100; } else { internal_frame_len = frame[1]; } if (internal_frame_len == (frame_len - 4)) { return 1; } else { return 0; } } static int frame_send(int tty_fd, unsigned char *frame, int frame_len) { int result, i; if (print_traffic) { printf(">>> "); for (i = 0; i < frame_len; i++) { printf("%02x ", frame[i]); } printf("\n"); } result = write(tty_fd, frame, frame_len); if (result == -1) { fprintf(stderr, "write() failed: %s\n", strerror(errno)); return -1; } return frame_len; } static int frame_recv(int tty_fd, unsigned char *frame, int frame_len_max) { int result, frame_len, checksum; unsigned char byte; if (print_traffic) printf("<<< "); frame_len = 0; while (1) { result = read(tty_fd, &byte, 1); if (result == -1) { if (errno != EAGAIN && errno != EWOULDBLOCK) { fprintf(stderr, "read() failed: %s\n", strerror(errno)); return -1; } } else if (result > 0) { if (print_traffic) printf("%02x ", byte); if (frame_len == frame_len_max) { fprintf(stderr, "frame_recv() failed: Overflow\n"); return -1; } frame[frame_len++] = byte; if (frame_is_complete(frame, frame_len)) { break; } } usleep(10); } if (print_traffic) printf("\n"); checksum = generate_checksum(&frame[2], frame[1]); if (checksum != frame[frame_len - 2]) { fprintf(stderr, "frame_recv() failed: Checksum incorrect\n"); return -1; } return frame_len; } static int command_baud_rate_set(int tty_fd) { int cmd_frame_len; unsigned char cmd_frame[8]; unsigned char status_frame[8]; cmd_frame_len = 0; cmd_frame[cmd_frame_len++] = 0x01; /* Command Frame Header */ cmd_frame[cmd_frame_len++] = 0x03; /* Command Information Length */ cmd_frame[cmd_frame_len++] = RL78_COMMAND_BAUD_RATE_SET; cmd_frame[cmd_frame_len++] = 0x00; /* Baud rate setting = 115200 */ cmd_frame[cmd_frame_len++] = 0x21; /* Voltage setting = 3.3V */ cmd_frame[cmd_frame_len++] = generate_checksum(&cmd_frame[2], cmd_frame[1]); cmd_frame[cmd_frame_len++] = 0x03; /* Command Frame Footer */ if (frame_send(tty_fd, cmd_frame, cmd_frame_len) < 0) { return -1; } if (frame_recv(tty_fd, status_frame, sizeof(status_frame)) < 0) { return -1; } if (status_frame[2] != RL78_STATUS_NORMAL_ACK) { fprintf(stderr, "command_baud_rate_set() failed: %s (0x%02x)\n", rl78_status_text(status_frame[2]), status_frame[2]); return -1; } if (print_details) { printf("Frequency: %d MHz\n", status_frame[3]); printf("Programming mode: %s\n", status_frame[4] ? "Wide-voltage" : "Full-speed"); } return 0; } static int command_reset(int tty_fd) { int cmd_frame_len; unsigned char cmd_frame[8]; unsigned char status_frame[8]; cmd_frame_len = 0; cmd_frame[cmd_frame_len++] = 0x01; /* Command Frame Header */ cmd_frame[cmd_frame_len++] = 0x01; /* Command Information Length */ cmd_frame[cmd_frame_len++] = RL78_COMMAND_RESET; cmd_frame[cmd_frame_len++] = generate_checksum(&cmd_frame[2], cmd_frame[1]); cmd_frame[cmd_frame_len++] = 0x03; /* Command Frame Footer */ if (frame_send(tty_fd, cmd_frame, cmd_frame_len) < 0) { return -1; } if (frame_recv(tty_fd, status_frame, sizeof(status_frame)) < 0) { return -1; } if (status_frame[2] != RL78_STATUS_NORMAL_ACK) { fprintf(stderr, "command_reset() failed: %s (0x%02x)\n", rl78_status_text(status_frame[2]), status_frame[2]); return -1; } return 0; } static int command_silicon_signature(int tty_fd) { int cmd_frame_len, status_frame_len, data_frame_len; unsigned char cmd_frame[8]; unsigned char status_frame[8]; unsigned char data_frame[32]; cmd_frame_len = 0; cmd_frame[cmd_frame_len++] = 0x01; /* Command Frame Header */ cmd_frame[cmd_frame_len++] = 0x01; /* Command Information Length */ cmd_frame[cmd_frame_len++] = RL78_COMMAND_SILICON_SIGNATURE; cmd_frame[cmd_frame_len++] = generate_checksum(&cmd_frame[2], cmd_frame[1]); cmd_frame[cmd_frame_len++] = 0x03; /* Command Frame Footer */ if (frame_send(tty_fd, cmd_frame, cmd_frame_len) < 0) { return -1; } if ((status_frame_len = frame_recv(tty_fd, status_frame, sizeof(status_frame))) < 0) { return -1; } if (status_frame[2] != RL78_STATUS_NORMAL_ACK) { fprintf(stderr, "command_silicon_signature() failed: %s (0x%02x)\n", rl78_status_text(status_frame[2]), status_frame[2]); return -1; } if ((data_frame_len = frame_recv(tty_fd, data_frame, sizeof(data_frame))) < 0) { return -1; } if (print_details) { printf("Device code: 0x%02x 0x%02x 0x%02x\n", data_frame[2], data_frame[3], data_frame[4]); printf("Device name: %c%c%c%c%c%c%c%c%c%c\n", data_frame[5], data_frame[6], data_frame[7], data_frame[8], data_frame[9], data_frame[10], data_frame[11], data_frame[12], data_frame[13], data_frame[14]); printf("Code flash ROM last address: 0x%02x%02x%02x\n", data_frame[17], data_frame[16], data_frame[15]); printf("Data flash ROM last address: 0x%02x%02x%02x\n", data_frame[20], data_frame[19], data_frame[18]); printf("Firmware version: %d.%d%d\n", data_frame[21], data_frame[22], data_frame[23]); } return 0; } static int command_block_erase(int tty_fd, int block_no) { int cmd_frame_len, status_frame_len, start_address; unsigned char cmd_frame[16]; unsigned char status_frame[8]; start_address = block_no * BLOCK_SIZE; cmd_frame_len = 0; cmd_frame[cmd_frame_len++] = 0x01; /* Command Frame Header */ cmd_frame[cmd_frame_len++] = 0x04; /* Command Information Length */ cmd_frame[cmd_frame_len++] = RL78_COMMAND_BLOCK_ERASE; cmd_frame[cmd_frame_len++] = (start_address & 0xff); /* Start Address, Low */ cmd_frame[cmd_frame_len++] = ((start_address >> 8) & 0xff); /* Start Address, Middle */ cmd_frame[cmd_frame_len++] = ((start_address >> 16) & 0xff); /* Start Address, High */ cmd_frame[cmd_frame_len++] = generate_checksum(&cmd_frame[2], cmd_frame[1]); cmd_frame[cmd_frame_len++] = 0x03; /* Command Frame Footer */ if (frame_send(tty_fd, cmd_frame, cmd_frame_len) < 0) { return -1; } if ((status_frame_len = frame_recv(tty_fd, status_frame, sizeof(status_frame))) < 0) { return -1; } if (status_frame[2] != RL78_STATUS_NORMAL_ACK) { fprintf(stderr, "command_block_erase() failed: %s (0x%02x)\n", rl78_status_text(status_frame[2]), status_frame[2]); return -1; } return 0; } static int command_programming(int tty_fd, int first_block_no, unsigned char *block_data, int no_of_blocks) { int cmd_frame_len, status_frame_len, data_frame_len, start_address, end_address, offset; unsigned char cmd_frame[16]; unsigned char status_frame[8]; unsigned char data_frame[264]; start_address = first_block_no * BLOCK_SIZE; end_address = ((first_block_no + no_of_blocks) * BLOCK_SIZE) - 1; cmd_frame_len = 0; cmd_frame[cmd_frame_len++] = 0x01; /* Command Frame Header */ cmd_frame[cmd_frame_len++] = 0x07; /* Command Information Length */ cmd_frame[cmd_frame_len++] = RL78_COMMAND_PROGRAMMING; cmd_frame[cmd_frame_len++] = (start_address & 0xff); /* Start Address, Low */ cmd_frame[cmd_frame_len++] = ((start_address >> 8) & 0xff); /* Start Address, Middle */ cmd_frame[cmd_frame_len++] = ((start_address >> 16) & 0xff); /* Start Address, High */ cmd_frame[cmd_frame_len++] = (end_address & 0xff); /* End Address, Low */ cmd_frame[cmd_frame_len++] = ((end_address >> 8) & 0xff); /* End Address, Middle */ cmd_frame[cmd_frame_len++] = ((end_address >> 16) & 0xff); /* End Address, High */ cmd_frame[cmd_frame_len++] = generate_checksum(&cmd_frame[2], cmd_frame[1]); cmd_frame[cmd_frame_len++] = 0x03; /* Command Frame Footer */ if (frame_send(tty_fd, cmd_frame, cmd_frame_len) < 0) { return -1; } if (frame_recv(tty_fd, status_frame, sizeof(status_frame)) < 0) { return -1; } if (status_frame[2] != RL78_STATUS_NORMAL_ACK) { fprintf(stderr, "command_programming() failed: %s (0x%02x)\n", rl78_status_text(status_frame[2]), status_frame[2]); return -1; } for (offset = 0; offset < (no_of_blocks * BLOCK_SIZE); offset += 256) { data_frame_len = 0; data_frame[data_frame_len++] = 0x02; /* Data Frame Header */ data_frame[data_frame_len++] = 0x00; /* Data Length, Always 0x00 = 256 Bytes */ memcpy(&data_frame[2], &block_data[offset], 256); data_frame_len += 256; data_frame[data_frame_len++] = generate_checksum(&data_frame[2], data_frame[1]); if ((offset + 256) >= (no_of_blocks * BLOCK_SIZE)) { data_frame[data_frame_len++] = 0x03; /* Data Frame Footer, End of Data */ } else { data_frame[data_frame_len++] = 0x17; /* Data Frame Footer, More To Be Sent */ } if (frame_send(tty_fd, data_frame, data_frame_len) < 0) { return -1; } if ((status_frame_len = frame_recv(tty_fd, status_frame, sizeof(status_frame))) < 0) { return -1; } if (status_frame[2] != RL78_STATUS_NORMAL_ACK) { fprintf(stderr, "command_programming() failed: %s (0x%02x)\n", rl78_status_text(status_frame[2]), status_frame[2]); return -1; } } if ((status_frame_len = frame_recv(tty_fd, status_frame, sizeof(status_frame))) < 0) { return -1; } if (status_frame[2] != RL78_STATUS_NORMAL_ACK) { fprintf(stderr, "command_programming() failed: %s (0x%02x)\n", rl78_status_text(status_frame[2]), status_frame[2]); return -1; } return 0; } static int command_checksum(int tty_fd, int first_block_no, int no_of_blocks) { int cmd_frame_len, status_frame_len, data_frame_len, start_address, end_address; unsigned char cmd_frame[16]; unsigned char status_frame[8]; unsigned char data_frame[8]; start_address = first_block_no * BLOCK_SIZE; end_address = ((first_block_no + no_of_blocks) * BLOCK_SIZE) - 1; cmd_frame_len = 0; cmd_frame[cmd_frame_len++] = 0x01; /* Command Frame Header */ cmd_frame[cmd_frame_len++] = 0x07; /* Command Information Length */ cmd_frame[cmd_frame_len++] = RL78_COMMAND_CHECKSUM; cmd_frame[cmd_frame_len++] = (start_address & 0xff); /* Start Address, Low */ cmd_frame[cmd_frame_len++] = ((start_address >> 8) & 0xff); /* Start Address, Middle */ cmd_frame[cmd_frame_len++] = ((start_address >> 16) & 0xff); /* Start Address, High */ cmd_frame[cmd_frame_len++] = (end_address & 0xff); /* End Address, Low */ cmd_frame[cmd_frame_len++] = ((end_address >> 8) & 0xff); /* End Address, Middle */ cmd_frame[cmd_frame_len++] = ((end_address >> 16) & 0xff); /* End Address, High */ cmd_frame[cmd_frame_len++] = generate_checksum(&cmd_frame[2], cmd_frame[1]); cmd_frame[cmd_frame_len++] = 0x03; /* Command Frame Footer */ if (frame_send(tty_fd, cmd_frame, cmd_frame_len) < 0) { return -1; } if ((status_frame_len = frame_recv(tty_fd, status_frame, sizeof(status_frame))) < 0) { return -1; } if (status_frame[2] != RL78_STATUS_NORMAL_ACK) { fprintf(stderr, "command_checksum() failed: %s (0x%02x)\n", rl78_status_text(status_frame[2]), status_frame[2]); return -1; } if ((data_frame_len = frame_recv(tty_fd, data_frame, sizeof(data_frame))) < 0) { return -1; } return data_frame[2] + (data_frame[3] * 0x100); } static int command_verify(int tty_fd, int first_block_no, unsigned char *block_data, int no_of_blocks) { int cmd_frame_len, status_frame_len, data_frame_len, start_address, end_address, offset; unsigned char cmd_frame[16]; unsigned char status_frame[8]; unsigned char data_frame[264]; start_address = first_block_no * BLOCK_SIZE; end_address = ((first_block_no + no_of_blocks) * BLOCK_SIZE) - 1; cmd_frame_len = 0; cmd_frame[cmd_frame_len++] = 0x01; /* Command Frame Header */ cmd_frame[cmd_frame_len++] = 0x07; /* Command Information Length */ cmd_frame[cmd_frame_len++] = RL78_COMMAND_VERIFY; cmd_frame[cmd_frame_len++] = (start_address & 0xff); /* Start Address, Low */ cmd_frame[cmd_frame_len++] = ((start_address >> 8) & 0xff); /* Start Address, Middle */ cmd_frame[cmd_frame_len++] = ((start_address >> 16) & 0xff); /* Start Address, High */ cmd_frame[cmd_frame_len++] = (end_address & 0xff); /* End Address, Low */ cmd_frame[cmd_frame_len++] = ((end_address >> 8) & 0xff); /* End Address, Middle */ cmd_frame[cmd_frame_len++] = ((end_address >> 16) & 0xff); /* End Address, High */ cmd_frame[cmd_frame_len++] = generate_checksum(&cmd_frame[2], cmd_frame[1]); cmd_frame[cmd_frame_len++] = 0x03; /* Command Frame Footer */ if (frame_send(tty_fd, cmd_frame, cmd_frame_len) < 0) { return -1; } if (frame_recv(tty_fd, status_frame, sizeof(status_frame)) < 0) { return -1; } if (status_frame[2] != RL78_STATUS_NORMAL_ACK) { fprintf(stderr, "command_verify() failed: %s (0x%02x)\n", rl78_status_text(status_frame[2]), status_frame[2]); return -1; } for (offset = 0; offset < (no_of_blocks * BLOCK_SIZE); offset += 256) { data_frame_len = 0; data_frame[data_frame_len++] = 0x02; /* Data Frame Header */ data_frame[data_frame_len++] = 0x00; /* Data Length, Always 0x00 = 256 Bytes */ memcpy(&data_frame[2], &block_data[offset], 256); data_frame_len += 256; data_frame[data_frame_len++] = generate_checksum(&data_frame[2], data_frame[1]); if ((offset + 256) >= (no_of_blocks * BLOCK_SIZE)) { data_frame[data_frame_len++] = 0x03; /* Data Frame Footer, End of Data */ } else { data_frame[data_frame_len++] = 0x17; /* Data Frame Footer, More To Be Sent */ } if (frame_send(tty_fd, data_frame, data_frame_len) < 0) { return -1; } if ((status_frame_len = frame_recv(tty_fd, status_frame, sizeof(status_frame))) < 0) { return -1; } if (status_frame[2] != RL78_STATUS_NORMAL_ACK) { fprintf(stderr, "command_verify() failed: %s (0x%02x)\n", rl78_status_text(status_frame[2]), status_frame[2]); return -1; } if (status_frame[3] != RL78_STATUS_NORMAL_ACK) { fprintf(stderr, "command_verify() failed: %s (0x%02x)\n", rl78_status_text(status_frame[3]), status_frame[3]); return -1; } } return 0; } static int command_block_blank_check(int tty_fd, int first_block_no, int no_of_blocks) { int cmd_frame_len, start_address, end_address; unsigned char cmd_frame[16]; unsigned char status_frame[8]; start_address = first_block_no * BLOCK_SIZE; end_address = ((first_block_no + no_of_blocks) * BLOCK_SIZE) - 1; cmd_frame_len = 0; cmd_frame[cmd_frame_len++] = 0x01; /* Command Frame Header */ cmd_frame[cmd_frame_len++] = 0x08; /* Command Information Length */ cmd_frame[cmd_frame_len++] = RL78_COMMAND_BLOCK_BLANK_CHECK; cmd_frame[cmd_frame_len++] = (start_address & 0xff); /* Start Address, Low */ cmd_frame[cmd_frame_len++] = ((start_address >> 8) & 0xff); /* Start Address, Middle */ cmd_frame[cmd_frame_len++] = ((start_address >> 16) & 0xff); /* Start Address, High */ cmd_frame[cmd_frame_len++] = (end_address & 0xff); /* End Address, Low */ cmd_frame[cmd_frame_len++] = ((end_address >> 8) & 0xff); /* End Address, Middle */ cmd_frame[cmd_frame_len++] = ((end_address >> 16) & 0xff); /* End Address, High */ cmd_frame[cmd_frame_len++] = 0x00; /* Specified Block */ cmd_frame[cmd_frame_len++] = generate_checksum(&cmd_frame[2], cmd_frame[1]); cmd_frame[cmd_frame_len++] = 0x03; /* Command Frame Footer */ if (frame_send(tty_fd, cmd_frame, cmd_frame_len) < 0) { return -1; } if (frame_recv(tty_fd, status_frame, sizeof(status_frame)) < 0) { return -1; } switch (status_frame[2]) { case RL78_STATUS_NORMAL_ACK: return 0; /* Blocks are free. */ case RL78_STATUS_IVERIFY_BLANK_ERROR: return 1; /* Blocks are occupied. */ default: fprintf(stderr, "command_block_blank_check() failed: %s (0x%02x)\n", rl78_status_text(status_frame[2]), status_frame[2]); return -1; } } static int programmer_init(char *tty_device) { int tty_fd, result; struct termios tio; unsigned int bits; tty_fd = open(tty_device, O_RDWR | O_NOCTTY | O_NONBLOCK); if (tty_fd == -1) { fprintf(stderr, "open(%s) failed: %s\n", tty_device, strerror(errno)); return -1; } memset(&tio, '\0', sizeof(tio)); tio.c_cflag = B115200 | CS8 | CSTOPB; tio.c_iflag = IGNPAR; tio.c_oflag = 0; tio.c_lflag = 0; result = tcsetattr(tty_fd, TCSANOW, &tio); if (result == -1) { fprintf(stderr, "tcsetattr() failed: %s\n", strerror(errno)); close(tty_fd); return -1; } result = ioctl(tty_fd, TIOCMGET, &bits); if (result == -1) { fprintf(stderr, "ioctl() failed: %s\n", strerror(errno)); close(tty_fd); return -1; } /* Set DTR (Reset Signal). */ bits |= TIOCM_DTR; result = ioctl(tty_fd, TIOCMSET, &bits); if (result == -1) { fprintf(stderr, "ioctl() failed: %s\n", strerror(errno)); close(tty_fd); return -1; } /* Turn on break. */ result = ioctl(tty_fd, TIOCSBRK, NULL); if (result == -1) { fprintf(stderr, "ioctl() failed: %s\n", strerror(errno)); close(tty_fd); return -1; } tcflush(tty_fd, TCIOFLUSH); /* Clear DTR (Reset Signal). */ bits &= (~TIOCM_DTR); result = ioctl(tty_fd, TIOCMSET, &bits); if (result == -1) { fprintf(stderr, "ioctl() failed: %s\n", strerror(errno)); close(tty_fd); return -1; } usleep(1000); /* Turn off break. */ result = ioctl(tty_fd, TIOCCBRK, NULL); if (result == -1) { fprintf(stderr, "ioctl() failed: %s\n", strerror(errno)); close(tty_fd); return -1; } tcflush(tty_fd, TCIOFLUSH); usleep(1000); /* Setup Two-wire UART mode. */ result = write(tty_fd, "\x00", 1); if (result == -1) { fprintf(stderr, "write() failed: %s\n", strerror(errno)); close(tty_fd); return -1; } else if (result == 0) { fprintf(stderr, "write() failed: Nothing written\n"); close(tty_fd); return -1; } usleep(1000); tcflush(tty_fd, TCIOFLUSH); return tty_fd; } static void programmer_shutdown(int tty_fd) { int result; unsigned int bits; result = ioctl(tty_fd, TIOCMGET, &bits); if (result == -1) { fprintf(stderr, "ioctl() failed: %s\n", strerror(errno)); } /* Set DTR (Reset Signal). */ bits |= TIOCM_DTR; result = ioctl(tty_fd, TIOCMSET, &bits); if (result == -1) { fprintf(stderr, "ioctl() failed: %s\n", strerror(errno)); } usleep(1000); /* Clear DTR (Reset Signal). */ bits &= (~TIOCM_DTR); result = ioctl(tty_fd, TIOCMSET, &bits); if (result == -1) { fprintf(stderr, "ioctl() failed: %s\n", strerror(errno)); } close(tty_fd); } static void display_help(char *progname) { fprintf(stderr, "Usage: %s <options>\n", progname); fprintf(stderr, "Options:\n" " -h Display this help and exit.\n" " -t Print TTY/serial traffic debugging info.\n" " -q Quiet mode, do not print anything.\n" " -v Verification mode, do not erase and program.\n" " -d DEVICE Use TTY DEVICE.\n" " -f FILE Use FILE for programming or verification.\n" " -o OFFSET Program or verify at block OFFSET instead of 0.\n" "\n"); } int main(int argc, char *argv[]) { int c, i, tty_fd, bin_len, block_no, checksum_remote, checksum_local, result; FILE *bin_fh; unsigned char bin_data[BLOCK_SIZE]; char *tty_device = NULL; char *bin_file = NULL; int mode_verify = 0; int block_offset = 0; while ((c = getopt(argc, argv, "htqvd:f:o:")) != -1) { switch (c) { case 'h': display_help(argv[0]); return EXIT_SUCCESS; case 't': print_traffic = 1; break; case 'q': print_traffic = 0; print_details = 0; break; case 'v': mode_verify = 1; break; case 'd': tty_device = optarg; break; case 'f': bin_file = optarg; break; case 'o': block_offset = atoi(optarg); break; case '?': default: display_help(argv[0]); return EXIT_FAILURE; } } if (tty_device == NULL) { fprintf(stderr, "Please specify a TTY!\n"); display_help(argv[0]); return EXIT_FAILURE; } if (bin_file == NULL) { fprintf(stderr, "Please specify a file!\n"); display_help(argv[0]); return EXIT_FAILURE; } tty_fd = programmer_init(tty_device); if (tty_fd == -1) { return EXIT_FAILURE; } if (command_baud_rate_set(tty_fd) != 0) { programmer_shutdown(tty_fd); return EXIT_FAILURE; } if (command_reset(tty_fd) != 0) { programmer_shutdown(tty_fd); return EXIT_FAILURE; } if (command_silicon_signature(tty_fd) != 0) { programmer_shutdown(tty_fd); return EXIT_FAILURE; } bin_fh = fopen(bin_file, "rb"); if (bin_fh == NULL) { fprintf(stderr, "fopen(%s) failed: %s\n", bin_file, strerror(errno)); programmer_shutdown(tty_fd); return EXIT_FAILURE; } checksum_local = 0; block_no = block_offset; while ((bin_len = fread(bin_data, sizeof(unsigned char), BLOCK_SIZE, bin_fh)) > 0) { if (print_details) { if (mode_verify == 0) { printf("Programming Block #%d (0x%06x -> 0x%06x)\n", block_no, (block_no * BLOCK_SIZE), (((block_no + 1) * BLOCK_SIZE) - 1)); } else { printf("Verifying Block #%d (0x%06x -> 0x%06x)\n", block_no, (block_no * BLOCK_SIZE), (((block_no + 1) * BLOCK_SIZE) - 1)); } } /* Pad remaining data with 0xff */ while (bin_len < BLOCK_SIZE) { bin_data[bin_len++] = 0xff; } for (i = 0; i < BLOCK_SIZE; i++) { checksum_local -= bin_data[i]; } if (mode_verify == 0) { result = command_block_blank_check(tty_fd, block_no, 1); if (result == -1) { fclose(bin_fh); programmer_shutdown(tty_fd); return EXIT_FAILURE; } else if (result == 1) { if (command_block_erase(tty_fd, block_no) != 0) { fclose(bin_fh); programmer_shutdown(tty_fd); return EXIT_FAILURE; } } if (command_programming(tty_fd, block_no, bin_data, 1) != 0) { fclose(bin_fh); programmer_shutdown(tty_fd); return EXIT_FAILURE; } } if (command_verify(tty_fd, block_no, bin_data, 1) != 0) { fclose(bin_fh); programmer_shutdown(tty_fd); return EXIT_FAILURE; } block_no += 1; } fclose(bin_fh); checksum_local = checksum_local & 0xffff; checksum_remote = command_checksum(tty_fd, block_offset, (block_no - block_offset)); if (print_details) { printf("Checksum Local : 0x%04x\n", checksum_local); printf("Checksum Remote: 0x%04x\n", checksum_remote); } programmer_shutdown(tty_fd); return EXIT_SUCCESS; }
Slackware 14.2 on a USB-stick
It's exactly 10 years since the first post on this website.
Anyway, I discovered that installing Slackware on a USB-stick wasn't as easy as it was last time in 2012, and those instructions are no longer valid for newer versions like 14.2. The main problem is that the kernel no longer contains built-in support for USB mass storage and the extended file systems.
Here are the updated steps:
1. Boot a host machine with the original Slackware DVD.
2. Insert the USB-stick into the host machine and use fdisk to create a large single Linux (0x83) partition on it.
3. Run the Slackware setup program and choose the newly created USB-stick partition as the target, with an ext4 file system.
4. Near the end of the setup process, skip the step involving installation of the LILO boot loader!
5. Unplug the USB-stick and put it into another fully functional Slackware 14.2 box.
6. Mount and chroot into the USB-stick filesystem. (In my case this was at /dev/sdc1 mounted on /mnt/sdc1):
mount /mnt/sdc1 mount -o bind /proc /mnt/sdc1/proc mount -o bind /sys /mnt/sdc1/sys mount -o bind /dev /mnt/sdc1/dev chroot /mnt/sdc1
6. Configure and update the Linux kernel, by following these steps:
cd /usr/src/linux make menuconfig # Do the necessary changes as described below. make make modules make modules_install make install # Will fail on LILO install, but just ignore this since kernel image is still copied to /boot.
In menuconfig, make sure to change these from module to built-in:
* xHCI HCD (USB 3.0) support
* EHCI HCD (USB 2.0) support
* USB Mass Storage support
* Second extended fs support
* The Extended 3 (ext3) filesystem
* The Extended 4 (ext4) filesystem
7. Modify /etc/fstab on the USB-stick and make sure that the root file system uses the correct partition device on the target machine. (e.g. /dev/sdb1)
8. Create the file /boot/extlinux.conf on the USB-stick and input contents similar to this:
default Linux prompt 1 timeout 100 label Linux kernel vmlinuz append root=/dev/sdb1 rootwait vga=normal vt.default_utf8=0
8. Exit the chroot environment:
exit umount /mnt/sdc1/proc umount /mnt/sdc1/sys umount /mnt/sdc1/dev
9. Install the boot loader on the USB-stick with the "extlinux" command like this: "extlinux -i /mnt/sdc1/boot". (Replace /mnt/sdc1 with correct mount point if necessary.)
10. Unmount the USB-stick and overwrite the master boot record on it with a command like this: "cat /usr/share/syslinux/mbr.bin > /dev/sdc". (Replace /dev/sdc with correct device if necessary.)
Simple File Selector in Curses
Here is a simple program that lets you select individual files from a tree view, and then the paths of the selected files are output to a file. The resulting file will typically be used as input for a completely different program.
Here is a screenshot:

Here is the code, which is based on the Directory Tree Diff Front End that I presented earlier:
#include <stdio.h> #include <stdlib.h> #include <string.h> #include <dirent.h> #include <sys/types.h> #include <sys/stat.h> #include <unistd.h> #include <curses.h> typedef enum { FILE_NODE_TYPE_ROOT, FILE_NODE_TYPE_DIR, FILE_NODE_TYPE_FILE, } file_node_type_t; typedef struct file_node_s { int marked; char *name; file_node_type_t type; struct file_node_s *parent; unsigned int no_of_subnodes; struct file_node_s **subnode; } file_node_t; static int curses_scroll_offset = 0; static int curses_selected_entry = 0; static file_node_t *file_node_new(file_node_t *parent, char *name, file_node_type_t type) { int len; file_node_t *new; new = (file_node_t *)malloc(sizeof(file_node_t)); if (new == NULL) return NULL; if (name == NULL) { new->name = NULL; } else { len = strlen(name) + 1; new->name = (char *)malloc(len); strncpy(new->name, name, len); } new->type = type; new->parent = parent; new->no_of_subnodes = 0; new->subnode = NULL; new->marked = 0; return new; } static file_node_t *file_node_add(file_node_t *current, char *name, file_node_type_t type) { file_node_t *new; new = file_node_new(current, name, type); if (new == NULL) return NULL; if (current->subnode == NULL) { current->subnode = malloc(sizeof(file_node_t *)); } else { current->subnode = realloc(current->subnode, sizeof(file_node_t *) * (current->no_of_subnodes + 1)); } current->subnode[current->no_of_subnodes] = new; current->no_of_subnodes++; return new; } static void file_node_remove(file_node_t *node) { int i; free(node->name); for (i = 0; i < node->no_of_subnodes; i++) { file_node_remove(node->subnode[i]); } if (node->subnode != NULL) { free(node->subnode); } free(node); } static int file_node_depth(file_node_t *node) { int depth; depth = 0; do { if (node->type == FILE_NODE_TYPE_ROOT) { break; } node = node->parent; depth++; } while (node != NULL); return depth; } static int file_node_list_size(file_node_t *node) { int i, size; if (node->type == FILE_NODE_TYPE_ROOT) { size = 0; } else { size = 1; } for (i = 0; i < node->no_of_subnodes; i++) { size += file_node_list_size(node->subnode[i]); } return size; } static char *file_node_path(file_node_t *node, char *path, int path_len) { char temp[PATH_MAX]; strncpy(path, node->name, path_len); node = node->parent; while (node != NULL) { if (node->type == FILE_NODE_TYPE_ROOT) { break; } strncpy(temp, path, PATH_MAX); snprintf(path, path_len, "%s/%s", node->name, temp); node = node->parent; } return path; } static int file_node_compare(const void *p1, const void *p2) { file_node_t *p1p, *p2p; p1p = *((file_node_t **)p1); p2p = *((file_node_t **)p2); return strcmp(((file_node_t *)p1p)->name, ((file_node_t *)p2p)->name); } static void file_node_sort(file_node_t *node) { int i; qsort(node->subnode, node->no_of_subnodes, sizeof(file_node_t *), file_node_compare); for (i = 0; i < node->no_of_subnodes; i++) { file_node_sort(node->subnode[i]); } } static void file_node_dump(file_node_t *node) { int i, depth; depth = file_node_depth(node); while (depth-- > 1) printf(" "); if (node->type == FILE_NODE_TYPE_ROOT) printf("/"); else { printf("%s", node->name); if (node->type == FILE_NODE_TYPE_DIR) printf("/"); } printf("\n"); for (i = 0; i < node->no_of_subnodes; i++) { file_node_dump(node->subnode[i]); } } static void file_node_print_marked(file_node_t *node, char *root_dir, FILE *fh) { int i; char path[PATH_MAX]; if (node->marked) { fprintf(fh, "%s/%s\n", root_dir, file_node_path(node, path, PATH_MAX)); } for (i = 0; i < node->no_of_subnodes; i++) { file_node_print_marked(node->subnode[i], root_dir, fh); } } static int file_node_scan(file_node_t *current, char *path) { DIR *dh; struct dirent *entry; struct stat st; char fullpath[PATH_MAX]; file_node_t *subnode; dh = opendir(path); if (dh == NULL) { fprintf(stderr, "Error: Unable to open directory: %s\n", path); return -1; } while ((entry = readdir(dh))) { if (entry->d_name[0] == '.') continue; /* Ignore files with leading dot. */ snprintf(fullpath, PATH_MAX, "%s/%s", path, entry->d_name); if (stat(fullpath, &st) == -1) { fprintf(stderr, "Warning: Unable to stat() path: %s\n", fullpath); continue; } if (S_ISDIR(st.st_mode)) { subnode = file_node_add(current, entry->d_name, FILE_NODE_TYPE_DIR); file_node_scan(subnode, fullpath); } else if (S_ISREG(st.st_mode)) { file_node_add(current, entry->d_name, FILE_NODE_TYPE_FILE); } } closedir(dh); return 0; } static file_node_t *file_node_get_by_node_no(file_node_t *node, int node_no, int *node_count) { file_node_t *found; int i; if (node->type != FILE_NODE_TYPE_ROOT) { *node_count = *node_count + 1; } if (*node_count == node_no) { return node; /* Match found, return self. */ } for (i = 0; i < node->no_of_subnodes; i++) { found = file_node_get_by_node_no(node->subnode[i], node_no, node_count); if (found != NULL) { return found; } } return NULL; } static void file_node_mark(file_node_t *node, int node_no) { int node_count; file_node_t *found; node_count = 0; found = file_node_get_by_node_no(node, node_no, &node_count); if (found == NULL) return; if (found->type != FILE_NODE_TYPE_FILE) return; /* Only files, not directories, can be marked. */ if (found->marked == 0) { found->marked = 1; } else { found->marked = 0; } } static void curses_list_draw(file_node_t *node, int line_no, int node_no, int selected) { int node_count, maxy, maxx, pos, depth; file_node_t *found; node_count = 0; found = file_node_get_by_node_no(node, node_no, &node_count); if (found == NULL) return; getmaxyx(stdscr, maxy, maxx); if (selected) attron(A_REVERSE); if (found->name == NULL) { for (pos = 0; pos < maxx - 2; pos++) mvaddch(line_no, pos, ' '); } else { pos = 0; /* Depth indicator. */ depth = file_node_depth(found); while (depth-- > 1) { mvaddch(line_no, pos++, ' '); mvaddch(line_no, pos++, ' '); } /* File/directory name. */ if (found->marked) attron(A_BOLD); mvaddstr(line_no, pos, found->name); pos += strlen(found->name); if (found->marked) attroff(A_BOLD); /* Slash for directory. */ if (found->type == FILE_NODE_TYPE_DIR) mvaddch(line_no, pos++, '/'); /* Padding. */ for (; pos < maxx - 2; pos++) mvaddch(line_no, pos, ' '); } if (selected) attroff(A_REVERSE); } static void curses_update_screen(file_node_t *node) { int n, i, maxy, maxx; int scrollbar_size, scrollbar_pos; int list_size; list_size = file_node_list_size(node); getmaxyx(stdscr, maxy, maxx); erase(); /* Draw text lines. */ for (n = 0; n < maxy; n++) { if ((n + curses_scroll_offset) >= list_size) break; if (n == (curses_selected_entry - curses_scroll_offset)) { curses_list_draw(node, n, n + curses_scroll_offset + 1, 1); } else { curses_list_draw(node, n, n + curses_scroll_offset + 1, 0); } } /* Draw scrollbar. */ if (list_size <= maxy) scrollbar_size = maxy; else scrollbar_size = maxy / (list_size / (double)maxy); scrollbar_pos = curses_selected_entry / (double)list_size * (maxy - scrollbar_size); attron(A_REVERSE); for (i = 0; i <= scrollbar_size; i++) mvaddch(i + scrollbar_pos, maxx - 1, ' '); attroff(A_REVERSE); mvvline(0, maxx - 2, 0, maxy); /* Place cursor at end of selected line. */ move(curses_selected_entry - curses_scroll_offset, maxx - 3); } static void curses_exit_handler(void) { endwin(); } static void curses_winch_handler(file_node_t *node) { endwin(); /* To get new window limits. */ curses_update_screen(node); flushinp(); keypad(stdscr, TRUE); } static void file_node_curses_loop(file_node_t *node) { int c, maxy, maxx, list_size; initscr(); atexit(curses_exit_handler); noecho(); keypad(stdscr, TRUE); while (1) { list_size = file_node_list_size(node); curses_update_screen(node); getmaxyx(stdscr, maxy, maxx); c = getch(); switch (c) { case KEY_RESIZE: curses_winch_handler(node); break; case KEY_UP: curses_selected_entry--; if (curses_selected_entry < 0) curses_selected_entry++; if (curses_scroll_offset > curses_selected_entry) { curses_scroll_offset--; if (curses_scroll_offset < 0) curses_scroll_offset = 0; } break; case KEY_NPAGE: curses_scroll_offset += maxy / 2; while (maxy + curses_scroll_offset > list_size) curses_scroll_offset--; if (curses_scroll_offset < 0) curses_scroll_offset = 0; if (curses_selected_entry < curses_scroll_offset) curses_selected_entry = curses_scroll_offset; break; case KEY_PPAGE: curses_scroll_offset -= maxy / 2; if (curses_scroll_offset < 0) curses_scroll_offset = 0; if (curses_selected_entry > maxy + curses_scroll_offset - 1) curses_selected_entry = maxy + curses_scroll_offset - 1; break; case ' ': case KEY_IC: file_node_mark(node, curses_selected_entry + 1); /* Move cursor to next line automatically. */ case KEY_ENTER: case '\n': case '\r': case KEY_DOWN: curses_selected_entry++; if (curses_selected_entry >= list_size) curses_selected_entry--; if (curses_selected_entry > maxy - 1) { curses_scroll_offset++; if (curses_scroll_offset > curses_selected_entry - maxy + 1) curses_scroll_offset--; } break; case '\e': /* Escape */ case 'Q': case 'q': return; } } } int main(int argc, char *argv[]) { file_node_t *root; char *root_dir; FILE *fh; if (argc < 2) { fprintf(stderr, "Usage: %s <output file> [directory]\n", argv[0]); return 1; } fh = fopen(argv[1], "wx"); if (fh == NULL) { fprintf(stderr, "Error: Cannot open file, or it exists already: %s\n", argv[1]); return 1; } if (argc > 2) { root_dir = argv[2]; } else { root_dir = "."; } root = file_node_new(NULL, NULL, FILE_NODE_TYPE_ROOT); if (root == NULL) { fclose(fh); return 1; } if (file_node_scan(root, root_dir) != 0) { file_node_remove(root); fclose(fh); return 1; } file_node_sort(root); if (isatty(STDOUT_FILENO)) { file_node_curses_loop(root); file_node_print_marked(root, root_dir, fh); } else { /* Mostly for debugging. */ file_node_dump(root); } file_node_remove(root); fclose(fh); return 0; }
4096 Byte Sector Mount
Using a large hard disk of over 2TB on a USB enclosure, and then attempting to use the same disk on a regular Serial ATA interface may not work at all. The issue lies in the USB enclosure using some trick to convert 512 byte sectors to 4096 byte sectors when displaying the disk to the OS. I present here the solution to mount and read such a disk with Linux.
I have used a 4TB disk with a small partition to experiment.
When the disk is connected to the USB enclosure, Linux reports it with 4096-byte logical blocks:
usb 2-1.2: new high speed USB device using ehci_hcd and address 4 usb 2-1.2: New USB device found, idVendor=174c, idProduct=5106 usb 2-1.2: New USB device strings: Mfr=2, Product=3, SerialNumber=1 usb 2-1.2: Product: USB to ATA/ATAPI bridge usb 2-1.2: Manufacturer: Asmedia usb 2-1.2: SerialNumber: 30700000000000000A22 scsi9 : usb-storage 2-1.2:1.0 scsi 9:0:0:0: Direct-Access WDC WD40 EZRZ-00WN9B0 80.0 PQ: 0 ANSI: 0 sd 9:0:0:0: Attached scsi generic sg6 type 0 sd 9:0:0:0: [sdf] 976754646 4096-byte logical blocks: (4.00 TB/3.63 TiB) sd 9:0:0:0: [sdf] Write Protect is off sd 9:0:0:0: [sdf] Mode Sense: 23 00 00 00 sd 9:0:0:0: [sdf] Assuming drive cache: write through
But when connected on the SATA interface, Linux reports it with 512-byte logical blocks:
ata6.00: ATA-9: WDC WD40EZRZ-00WN9B0, 80.00A80, max UDMA/133 ata6.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 0/32) ata6.00: configured for UDMA/100 ata6: EH complete scsi 5:0:0:0: Direct-Access ATA WDC WD40EZRZ-00W 0A80 PQ: 0 ANSI: 5 sd 5:0:0:0: [sdd] 7814037168 512-byte logical blocks: (4.00 TB/3.64 TiB) sd 5:0:0:0: [sdd] 4096-byte physical blocks sd 5:0:0:0: [sdd] Write Protect is off sd 5:0:0:0: [sdd] Mode Sense: 00 3a 00 00 sd 5:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Attempting to mount the partition directly will fail because of the sector (block) mismatch:
# mount /dev/sdd1 /mnt/sdd1 mount: /dev/sdd1 is write-protected, mounting read-only NTFS signature is missing. Failed to mount '/dev/sdd1': Invalid argument The device '/dev/sdd1' doesn't seem to have a valid NTFS. Maybe the wrong device is used? Or the whole disk instead of a partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?
To get around this, we need to use a loopback device, but first some calculations must be done, based on the start and end sectors of the partition. This data can be gotten with e.g. fdisk:
# fdisk -l /dev/sdd Disk /dev/sdd: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: dos Disk identifier: 0x52205024 Device Boot Start End Sectors Size Id Type /dev/sdd1 256 25855 25600 12.5M 83 Linux
Using 4096 byte sectors, we get the start sector at byte: 4096 * 256 = 1048576
And a byte size of: 4096 * 25600 = 104857600
Using these numbers, the partition can be mounted like so:
losetup --verbose --offset 1048576 --sizelimit 104857600 /dev/loop0 /dev/sdd mount /dev/loop0 /mnt/loop
And after use, unmounted like so:
umount /mnt/loop losetup -d /dev/loop0
Recursive String Search Bugfix
Remember Recursive String Search Improved?
It actually contains a very subtle bug, which I discovered after a search I made did not make sense. Something that should have been found wasn't.
As an example, it turns out that searching for "obar" did not locate the string "Foobar". The current algorithm would find the "o" in the string first, then the other "o", but since this is not what was searched for, the search was stopped. It did not backtrack in the original string, which it should have, and this is what has been fixed.
Only a single line of code was added:
#include <stdlib.h> #include <stdio.h> #include <string.h> #include <ctype.h> #include <sys/types.h> #include <sys/stat.h> #include <limits.h> #include <unistd.h> #include <dirent.h> #define FILTER_DELIMITER ";" #define FILTER_MAX 10 static char *filters[FILTER_MAX]; static int no_of_filters = 0; static void display_help(char *progname) { fprintf(stderr, "Usage: %s <options> <string>\n", progname); fprintf(stderr, "Options:\n" " -h Display this help and exit.\n" " -n Print filename on each search match line.\n" " -d DIR Search DIR instead of current directory.\n" " -f FILTER Apply FILTER on filename extension when searching.\n" " Delimited by ';', e.g. '.c;.cpp;.h;.hpp'\n" "\n"); } static void build_filter(char *filter) { no_of_filters = 0; filters[0] = strtok(filter, FILTER_DELIMITER); if (filters[0] == NULL) return; for (no_of_filters = 1; no_of_filters < FILTER_MAX; no_of_filters++) { filters[no_of_filters] = strtok(NULL, FILTER_DELIMITER); if (filters[no_of_filters] == NULL) break; } } static int matches_filter(char *name, int name_len) { int i, n1, n2, match, filter_len; if (no_of_filters == 0) return 1; /* No filters, always matches. */ for (i = 0; i < no_of_filters; i++) { filter_len = strlen(filters[i]); if (filter_len > name_len) return 0; /* Filter cannot be longer than name! */ match = 0; n2 = name_len - 1; for (n1 = filter_len - 1; n1 >= 0; n1--, n2--) { if (toupper(filters[i][n1]) != toupper(name[n2])) break; match++; } if (filter_len == match) return 1; /* Whole filter matched. */ } return 0; /* No matches. */ } static void recurse(char *path, char *string, int string_len, int show_mode) { static char line[1024]; /* Allocate in BSS. */ char full_path[PATH_MAX]; int n, match, name_shown, line_no; struct dirent *entry; struct stat st; DIR *dh; FILE *fh; dh = opendir(path); if (dh == NULL) { fprintf(stderr, "Warning: Unable to open directory: %s\n", path); return; } while ((entry = readdir(dh))) { if (entry->d_name[0] == '.') continue; /* Ignore files with leading dot. */ #ifdef WINNT snprintf(full_path, PATH_MAX, "%s\\%s", path, entry->d_name); #else snprintf(full_path, PATH_MAX, "%s/%s", path, entry->d_name); #endif stat(full_path, &st); if (S_ISDIR(st.st_mode)) { /* Traverse. */ recurse(full_path, string, string_len, show_mode); } else if (S_ISREG(st.st_mode)) { /* Search. */ if (! matches_filter(full_path, strlen(full_path))) continue; fh = fopen(full_path, "r"); if (fh == NULL) { fprintf(stderr, "Warning: Unable to open file: %s\n", full_path); continue; } name_shown = line_no = 0; while (fgets(line, 1024, fh) != NULL) { line_no++; match = 0; for (n = 0; line[n] != '\0'; n++) { if (toupper(line[n]) == toupper(string[match])) { match++; if (match >= string_len) { if (show_mode == 0) { if (! name_shown) { printf("%s\n", full_path); name_shown = 1; } printf("%d:%s", line_no, line); } else { printf("%s:%d:%s", full_path, line_no, line); } break; } } else { n -= match; /* Backtrack */ match = 0; } } } fclose(fh); } } closedir(dh); return; } int main(int argc, char *argv[]) { int c; int show_mode = 0; char *search_dir = NULL; while ((c = getopt(argc, argv, "hnd:f:")) != -1) { switch (c) { case 'h': display_help(argv[0]); exit(EXIT_SUCCESS); case 'n': show_mode = 1; break; case 'd': search_dir = optarg; break; case 'f': build_filter(optarg); break; case '?': default: display_help(argv[0]); exit(EXIT_FAILURE); } } if (search_dir == NULL) { search_dir = "."; /* Current directory. */ } if (optind >= argc) { display_help(argv[0]); return EXIT_FAILURE; } recurse(search_dir, argv[optind], strlen(argv[optind]), show_mode); return EXIT_SUCCESS; }
Here is also a binary for Win32 (compiled with MinGW).
Libvirt for KVM Guest
Referring to the Buildroot-based KVM Guest earlier, here is a way to set it up using libvirt: The virtualization API. Using libvirt instead of just QEMU gives additional flexibility, like easier pinning of physical CPUs to VCPUs and so on.
This setup also assumes that the root filesystem is already laid out in /tmp/kvm_guest, to be shared using the 9P protocol. Due to the fact that many files in the root filesystem are owned by root, the whole virtualization also needs to be run as root, or else the host will not allow access to the shared files.
Here is the XML configuration file for libvirt, some paths and the UUID may need to be changed. I named it "kvm_guest.xml":
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>kvm_guest</name> <uuid>00000000-0000-0000-0000-000000000000</uuid> <memory unit='KiB'>131072</memory> <currentMemory unit='KiB'>131072</currentMemory> <vcpu placement='static'>1</vcpu> <os> <type arch='x86_64' machine='pc-i440fx-2.6'>hvm</type> <kernel>/tmp/buildroot/kvmguest/output/images/bzImage</kernel> <cmdline>root=/dev/root rw rootfstype=9p rootflags=trans=virtio console=hvc0</cmdline> </os> <features> <acpi/> </features> <cpu mode='custom' match='exact'> <model fallback='allow'>kvm64</model> </cpu> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <controller type='pci' index='0' model='pci-root'/> <memballoon model='none'/> <console type='pty'> <target type='virtio' port='0'/> </console> <interface type='ethernet'> <mac address='00:00:de:ad:be:ef'/> <target dev='tap0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> </devices> <qemu:commandline> <qemu:arg value='-chardev'/> <qemu:arg value='stdio,id=char0'/> <qemu:arg value='-device'/> <qemu:arg value='virtio-serial'/> <qemu:arg value='-device'/> <qemu:arg value='virtconsole,chardev=char0'/> <qemu:arg value='-fsdev'/> <qemu:arg value='local,id=fs0,path=/tmp/kvm_guest,security_model=none'/> <qemu:arg value='-device'/> <qemu:arg value='virtio-9p-pci,fsdev=fs0,mount_tag=/dev/root'/> </qemu:commandline> </domain>
It can then be started and accessed like so:
sudo /etc/rc.d/rc.libvirt start sudo virsh create kvm_guest.xml sudo virsh console kvm_guest
The rc.libvirt script is possibly Slackware specific, your system may have a different way to start the libvirt daemons.
Bluray on Slackware
Here's information on how I was able to play encrypted Bluray disks on Slackware using a custom MPlayer.
This statement from the MPlayer homepage is vital:
"MPlayer does support encrypted BluRay playback, though not all steps are handled by MPlayer itself. The two alternative methods use the URL schemes bd:// (always supports decryption, but you need the key for each and every disk in ~/.dvdcss/KEYDB.cfg and only works well with very simple BluRays, similar to dvd:// vs. dvdnav://) and br:// (uses libbluray and should support the same as VideoLAN in the link below but that is untested)."
I only got it to work with the "br://" scheme. The MPlayer package that follows the default Slackware installation does NOT link to libbluray, so I had to download and compile it manually, and then use this separate MPlayer binary for Bluray playback only. Prior to this, I had already installed libbluray, libbdplus and libaacs on the system.
I launch it with the following helper script:
#!/bin/sh if ! fgrep /mnt/cdrom /proc/mounts > /dev/null; then mount /mnt/cdrom || exit 1 fi # Enable debugging info for libbluray: export BD_DEBUG_MASK="0xFFFFFFF" # NOTE: Set the BLURAY_TITLE variable to change titles. # NOTE: Use -chapter <id> argument to change chapters. export BLURAY_TITLE=0 exec ~/opt/mplayer-bluray-bin br://$BLURAY_TITLE//mnt/cdrom "$@"
Finally, you will need to have an updated KEYDB.cfg file at ~/.config/aacs/KEYDB.cfg and hope that the VUK (Volume Unique Key) has been discovered for your Bluray disc.
Buildroot for KVM Guest
Here is a set of configuration files to quickly build a minimal KVM-based hypervisor guest with Buildroot. It attempts to use mostly VirtIO paravirtualization to maximize performance. This includes virtual console, virtual network and virtualized filesystem with the 9P protocol.
These files are based on version 2016.11 of Buildroot, gotten from https://buildroot.org/downloads/buildroot-2016.11.tar.bz2
Main Buildroot configuration file, should be stored as "configs/kvm_guest_defconfig" in the Buildroot structure:
BR2_x86_64=y BR2_x86_core2=y BR2_TOOLCHAIN_BUILDROOT_GLIBC=y BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_4=y BR2_TOOLCHAIN_BUILDROOT_CXX=y BR2_TARGET_GENERIC_HOSTNAME="kvm-guest" BR2_TARGET_GENERIC_ISSUE="Welcome to Virtualized KVM Guest" BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV=y BR2_TARGET_GENERIC_GETTY_PORT="hvc0" BR2_TARGET_GENERIC_GETTY_BAUDRATE_115200=y BR2_LINUX_KERNEL=y BR2_LINUX_KERNEL_CUSTOM_VERSION=y BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.4.38" BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/kvm_guest/linux-4.4.config" BR2_LINUX_KERNEL_INSTALL_TARGET=y BR2_TARGET_ROOTFS_EXT2=y BR2_TARGET_ROOTFS_EXT2_4=y # BR2_TARGET_ROOTFS_TAR is not set
Linux kernel configuration, should be stored as "board/kvm_guest/linux-4.4.config":
CONFIG_SYSVIPC=y CONFIG_NO_HZ_IDLE=y CONFIG_HIGH_RES_TIMERS=y CONFIG_VIRT_CPU_ACCOUNTING_GEN=y CONFIG_NAMESPACES=y CONFIG_KALLSYMS_ALL=y CONFIG_EMBEDDED=y CONFIG_MODULES=y CONFIG_MODULE_UNLOAD=y CONFIG_SMP=y CONFIG_HYPERVISOR_GUEST=y CONFIG_SCHED_SMT=y CONFIG_NUMA=y CONFIG_IA32_EMULATION=y CONFIG_NET=y CONFIG_PACKET=y CONFIG_UNIX=y CONFIG_INET=y CONFIG_NET_9P=y CONFIG_NET_9P_VIRTIO=y CONFIG_DEVTMPFS=y CONFIG_DEVTMPFS_MOUNT=y CONFIG_VIRTIO_BLK=y CONFIG_NETDEVICES=y CONFIG_VIRTIO_NET=y CONFIG_VT_HW_CONSOLE_BINDING=y CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_VIRTIO_CONSOLE=y CONFIG_HW_RANDOM=y CONFIG_HW_RANDOM_VIRTIO=y CONFIG_VIRT_DRIVERS=y CONFIG_VIRTIO_PCI=y CONFIG_VIRTIO_BALLOON=y CONFIG_VIRTIO_INPUT=y CONFIG_VIRTIO_MMIO=y CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y CONFIG_EXT2_FS=y CONFIG_EXT3_FS=y CONFIG_TMPFS=y CONFIG_TMPFS_POSIX_ACL=y CONFIG_9P_FS=y CONFIG_9P_FS_POSIX_ACL=y CONFIG_DEBUG_FS=y CONFIG_STACKTRACE=y
Once the files are in place, the build can be performed with:
make kvm_guest_defconfig make
Here is a helper script to unpack the root filesystem and launch the whole thing under QEMU:
#!/bin/sh if [ `id -u` != "0" ]; then echo "Must be root!" exit 1 fi cd `dirname "$0"` if [ ! -d /tmp/kvm_guest ]; then mkdir -p /tmp/kvm_guest mkdir -p /tmp/kvm_guest_loop mount -o loop output/images/rootfs.ext4 /tmp/kvm_guest_loop cp -a /tmp/kvm_guest_loop/* /tmp/kvm_guest umount /tmp/kvm_guest_loop rmdir /tmp/kvm_guest_loop fi qemu-system-x86_64 \ -enable-kvm \ -cpu host \ -m 128 \ -kernel output/images/bzImage \ -serial none \ -display none \ -monitor none \ -chardev stdio,id=char0 \ -device virtio-serial \ -device virtconsole,chardev=char0 \ -fsdev local,id=fs0,path=/tmp/kvm_guest,security_model=none \ -device virtio-9p-pci,fsdev=fs0,mount_tag=/dev/root \ -net nic,macaddr=00:00:de:ad:be:ef,model=virtio,vlan=1 \ -net tap,ifname=tap0,vlan=1,script=no,downscript=no \ -append "root=/dev/root rw rootfstype=9p rootflags=trans=virtio console=hvc0"