Battery power bank, WiFi USB stick, and thumb drive

I then turn away to motor-less small storage, a 32 Gigs USB thumb drive so I can move on to other concern: on-the-fly encryption. Hence, I follow instructions to install True Crypt on Raspberry Pi. After unpacking of wxWidgets-2.8.12.tar.gz and TrueCrypt 7.1a Source.tar.gz in place, putting header files from pkcs-11-cryptoki2.20, and then install libfuse-dev, the following make will require long time:

$ export PKCS11_INC=/usr/local/src/truecrypt/pkcs/
 
$ make NOGUI=1 WX_ROOT=/usr/src/wxWidgets wxbuild
Configuring wxWidgets library...
Building wxWidgets library...
/usr/src/wxWidgets/src/common/string.cpp:84:39: warning: ‘wxEmptyString’ initialized and declared ‘extern’ [enabled by default]

$ make NOGUI=1 WX_ROOT=/usr/src/wxWidgets wxbuild
Compiling Buffer.cpp
Compiling Exception.cpp
Compiling Event.cpp
...
../Crypto/Aeskey.c:527:25: warning: operation on ‘ss[7]’ may be undefined [-Wsequence-point]
../Crypto/Aeskey.c:527:25: warning: operation on ‘ss[7]’ may be undefined [-Wsequence-point]
../Crypto/Aeskey.c:527:25: warning: operation on ‘ss[7]’ may be undefined [-Wsequence-point]
...
Converting Language.xml
Compiling Resources.cpp
Linking truecrypt

I created the TrueCrypt volume separately via its desktop GUI with ext3 file system to then mount it in Pi to a configured Samba share. As pointed out in a post, the following changes are added to smb.conf

...
wins support = yes
...
[pitruecrypt]
   comment= Pi Truecrypt Volume
   path=<the mount path of the USB thumb drive TrueCrypt volume>
   browseable=Yes
   writeable=Yes
   only guest=no
   create mask=0777
   directory mask=0777
   public=no

and then user-password are entered via interactive command.

For mobility, I already had the Pi as WiFi access point using hostapd (check these steps) and power bank, so it’s now matter of performance. In the case of rsync, initial sync of some 1,500 items totaling in 1 Gig size elapses in approximately the same 12 minutes of time compared to one bulk file of the same size. Of course, over the next incremental sync, it only takes less than a minute for the thousand items to just update slight differences.

Security

Back to security, there surely risk by opening Samba share to mounted TrueCrypt volume. But, for me it would be practically manageable (cross my finger). There is more concern to the fate of TrueCrypt after it is being closed in such a weird way, given that last audit finds nothing severe. Anyway, I found brute force tool, but no critical attack exists currently, unless e.g. it stays powered on and mounted, the person gain physical access. Beats me again.

A matrix keypad instance is defined by:

1. Actual GPIO pins used forming the row and column of the m x n matrix
2. Individual character in-use as symbol for each button

Hence, I instantiate and call method like

...
QPad  = matrixQPi(keyPad=keyPad,row=row,col=col)
print QPad.scanQ()

to print the character being pressed. Some examples pushed to my github explains how the above keyPad, row, and col are defined to scan-read pressed button of 2×2, 2×3, and 4×3 matrix keypads (or have it 3×4 matrix keypads in other words).

Illustration of any m-by-n matrix: 4x3, 2x2, and 2x3 keypad with different button symbols & GPIO combinations

I used deprecated Wiring-Pi Python (they already moved to 2.x version) without problem. However, you’ll fail building from latest commit and must use combination of older commits as described by my updated part of an old-post. By the way, there’s I/O expander support for WiringPi2-Python which is good, considering:

GPIO is expensive and for the sake of a keypad, you should not spend all.

(A friend told me that once)

In the Debian example, the wiki introduced NetInstall. Later I find out the mk_mmc.sh script shown there to be useful for (1) duplicating and (2) restoring working Linux backup to new or corrupted MMC/SD card. I break down that Robert C. Nelson’s mk_mmc.sh script at GitHub to small routines for setting up the card.

Typical boot-rootfs partitions of SD card to boot ARM-Linux boards

It’s an elegant script automating the process of preparing SD card for embedded Linux, but requires large downloading on the run (mostly root file system and then U-Boot binaries and loader config). Duplicating/restoring means we already have those and probably only need to do elementary process rather than full card preparation. Below are some processes that can be repeatedly used.

Checking Tool Versions

The tools to work with are common in Linux distribution (I used Ubuntu) and require root privilege. The script checked for fdisk and parted installed. My versions are

$ fdisk -v
fdisk (util-linux-ng 2.17.2)
$ parted -v
parted (GNU parted) 2.2

Originally mk_mmc.sh will exit If “GNU Fdisk” found instead. Also for the above parted version, “--align cylinder” argument will be used.

Creating Two Partitions

Setting up two partitions which are the boot image (FAT 16) and root file system (ext4). The SD card plugged in through USB card reader found as /dev/sdb in this example (we’re formatting be sure of this!). If you plug MMC card to laptop slot, it may be found as /dev/mmcblk0.

1. Format card into a single disk

$ parted --script /dev/sdb mklabel msdos
$ fdisk -l /dev/sdb

2. Create the boot partition (allocating 64MB)

$ fdisk /dev/sdb << END
n
p
1
1
+64M
t
e
p
w
END

then make sure it’s done.

$ sync

3. Flag it as boot

$ parted --script /dev/sdb set 1 boot on

4. Create the root file system by first determining space left after the first partition

$ END_BOOT=$(LC_ALL=C parted -s /dev/sdb unit mb print free | grep primary | awk '{print $3}' | cut -d "M" -f1)
$ echo $END_BOOT 
69.7
$ END_DEVICE=$(LC_ALL=C parted -s /dev/sdb unit mb print free | grep Free | tail -n 1 | awk '{print $2}' | cut -d "M" -f1)
$ echo $END_DEVICE 
3951

In this case 4GB SD card is used and the above calculation will stretch space left from 69.7 to 3951 of as rootfs partition.

$ parted --script --align cylinder /dev/sdb mkpart primary ext4 $END_BOOT $END_DEVICE

5. Format boot and rootfs according to each file system type with those naming also. They’re already accessible as /dev/sdb1 and /dev/sdb2 (previously didn’t exist) or relevant /dev/mmcblk0p(some index).

$ mkfs.vfat -F 16 /dev/sdb1 -n boot
$ mkfs.ext4 /dev/sdb2 -L rootfs

6. Finally we have something like the output of parted below

$ LC_ALL=C parted -s /dev/sdb unit mb print free 
Model: HUAWEI MMC Storage (scsi)
Disk /dev/sdb: 4023MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
 
Number  Start   End     Size    Type     File system  Flags
 1      0.03MB  70.9MB  70.8MB  primary  fat16        boot, lba
        70.9MB  4023MB  3953MB           Free Space

(yes I used Huawei modem as the microSD card reader)

Restoring Linux

For the boot partition content we can simply copy-paste the backup files, while for rootfs normally compressed file (preserving path information) is used for backup. Restore this by mounting rootfs and then unpacking the file (again /dev/sdb2 in this case).

[root@host]$ mkdir /media/rootfs
[root@host]$ mount -t ext4 /dev/sdb2 /media/rootfs
[root@host]$ pv armel-rootfs-201110131018.tar | sudo tar --numeric-owner --preserve-permissions -xf - -C /media/rootfs

Below is an example of how to backup rootfs from the mounted SD card (/media/rootfs).

[root@host]$ cd /media/rootfs
[root@host]$ /media/rootfs# tar --numeric-owner --preserve-permissions -cvf /home/arif/Documents/arm-linux/ubuntu/rootstock-rootfs/armel-rootfs-201202141421.tar ./*

(Check my other post related to creating rootfs)

Creating Swap

mk_mmc.sh also offers optional swap creation. It is a “file” inside rootfs partition created by

[root@host]$ dd if=/dev/zero of=/media/rootfs/SWAP.swap bs=1M count=250
[root@host]$ mkswap /media/rootfs/SWAP.swap

(the above will create 262MB swap inside rootfs SD card currently mounted to /media/rootfs of the laptop)

It will be mounted as swap by the ARM-Linux board via fstab setting configured below.

[root@host]$ echo "/SWAP.swap  none  swap  sw  0 0" >> /media/rootfs/etc/fstab

Synchronize and safely remove the card

[root@host]$ sync
[root@host]$ umount /media/rootfs

Now it’s almost automatically attached as modem after insertion and people have been contributing to list of device-and-target device after mode-switching (find it as /usr/share/usb_modeswitch/configPack.tar.gz). At least for Raspberry Pi (RPi), I have one device list from August 2012 when playing around with XBian 0.8.3 and one from May 2012 in when using Raspbian Wheezy (2012-08-16).

Huawei E220 requires no mode-switch from vendor-product ID 12d1:1003 to function as modem

There is still wrapper for udev in /lib/udev/rules.d/40-usb_modeswitch.rule and my Huawei E153 HSDPA stick recognized and switched successfully as shown:

usb_modeswitch: switching device 12d1:1446 on 001/005
...
logger: usb_modeswitch: switched to 12d1:14ac on 001/006

But I didn’t get there in the first place. Because of USB power problem (detailed in previous post), I used to plug USB devices before powering up so that I could use Y-cable injected by separate power source without powered USB-hub. It turned out that if I had plugged the USB before powering up the RPi the mode-switching didn’t occur.

The HSDPA USB modem used is Huawei E153 appearing in lsusb with vendor-product ID 12d1:1446. Using aforementioned XBian and target device 12d1:140c (suggested in Santinoli’s post), I arbitrarily succeeded to switch to modem and dial internet with pppd.

[root]$ usb_modeswitch -v 12d1 -p 1446 -P 140c

(I placed Santinoli’s config in /etc/usb_modeswitch.conf)
This wasn’t stable however (also note that XBian consumed CPU above 50% for xbmc).

Later I find out that it will automatically switch to 12d1:14ac instead when plugged in after RPi is up. However, since it also won’t switch if plugged before power-up, I also try to mode-switch manually using config file found inside /usr/share/usb_modeswitch/configPack.tar.gz archive named 12d1:1446.

[root]$ usb_modeswitch -v 12d1 -p 1446 -c 12d1\:1446

It doesn’t seem to work. Modification to something close to my previous config also fails. Hence, consistent result only occurs if the stick is plugged in after RPi up. Then I decide to use USB-hub (also detailed in previous post) to tackle power issue (it will reboot anyway if you plug in without adequate powering). However, after successful mode-switch what’s being reported in eLinux Wiki often happens: it slows down, try lsub command for example.

Currently I give up the idea of using that modem and choose older Huawei E220:

1. No usb_modeswitch. Driver attached directly by the kernel as option.ko device (this device has been on the list for long)
2. Stable. No slowing down

The E153 modem itself has been implemented to stream video using the higher ARM type BeagleBoard-xM without significant stability issue (check my YouTube demo). It mode-switched when powering up Ubuntu on the Beagle.

Y-cable for USB modem stick & various current rating-AC power adapters for smartphone/tablet to go with Raspberry Pi

eLinux Wiki lists market available USB-hubs reported to work. A decent brand (that will cost you) qualifies:

• Doesn’t leak power back to RPi via USB port (check this by unplugging the main power, then check the red LED indicator) as this will interfere with the reboot command (physical state vs software).
• Provides charging from one of its port for RPi, meaning no separate source for the exclusive 700 mA requirement. A single AC power adapter with high current rating will fit all (check single power connectivity sketch in this blog post)

An ugly situation when I plugged in WiFi stick and modem looks like:

kernel: [ 1836.871640] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000118
...
kernel: [ 1803.780601] phy0 -> rt2x00usb_vendor_request: Error - Vendor Request 0x07 failed for offset 0x101c with error -110.

tail of the above syslog is saying error with the (1) ethernet (this is actually also a USB 2.0 but not appearing in lsusb command) and (2) the WiFi stick used (in this case with RT5370 chipset). Then, of course (3) doing things with the just plugged modem you’ll expect more instability.

I mentioned BeagleBoard-xM which also came with similar USB ethernet on-board. Unfortunately, Beagle was more stable when my case was to have it stream a zoom-camera input to internet (check my YouTube demo). The camera decoder and modem were both USB plugged without Y-cable or additional power supply. As a standalone system, part of the design was to reboot under trouble detected by the software. Again, OS command reboot is not achievable consistently with the RPi when power backfeeds through USB port.

(PS: Note that previous comparison solely points out what to expect when dealing with USB among other considerations e.g. price, different ARM, etc. which are entirely different.)

As an absolute requirement for me, I found a powered USB-hub available in Indonesian market as 7 ports XTec Go. Yes, it leaks power back as confirmed by the vendor-product ID 05e3:0608 of the chipset listed in eLinux Wiki lists with a bunch of different names (judging the picture, Hama is closest).

$ lsusb
...
Bus 001 Device 004: ID 05e3:0608 Genesys Logic, Inc. USB-2.0 4-Port HUB
Bus 001 Device 005: ID 05e3:0608 Genesys Logic, Inc. USB-2.0 4-Port HUB
...

Three Musketeers: powered USB-hub (Genesys chipset), Huawei E220 HSDPA modem, and TL-WN727N WiFi stick

My el cheapo USB-hub, HDSPA modem, and WiFi stick are the matchmaking of brands shown in the above image. It achieves long uptime like 6 hours serving internet when RPi used as router. I still use separate AC power adapter on the go with me: regular 700 mA that comes with smartphone will do for the RPi, while for the USB-hub, I have 2A rating from tablet charger. However, I’ve also plugged separate 500 mA charger for the hub without problem and/or excessive heat (you need to touch the supply sometime to feel any nasty heat as some of them might have been low quality build that could blow up. Yes it did). My plan with mobility is to have power bank with dual USB charging output (this Hame has 1A and 2.1A) when no AC source nears.

Have I tested other USB modem stick? Check my other post related to usb_modeswitch.

Updated: for those who fail to build using deprecated WiringPi-Python, check this updated post to know which commit that build without error. There is now also a Python class for matrix keypad.

My idea of having the keypad is to make alternative input available under no keyboard presence nor shell access. List of things I can think of for instances, pressed key “0″ will make the Raspberry Pi (RPi) dial GPRS to a specific ISP and act as router to the USB WiFi stick, pressed key “7″ will convert it to a router that will bridge the ethernet to WiFi, etc. In short, those key readings will invoke subsequent scripts to run inside RPi.

The physical connection schematic drawings and code are gitified (visit v1.2 of the project on gitHub). A nice animated image on how the buttons connect pins forming a matrix can be found in hackyourmind.org.

Matrix keypad: alternative quick input for Raspberry Pi to start certain command

Some remarks over what the code does:

• It only read one pressed key at a time.
• It uses no interrupt. To read input it must wait and scan (this waiting experience is by far negligible to human sense).
• Debouncing doesn’t seem necessary as WiringPi already provided this by software (I suppose this timing method on their gitHub deals with debounce).

How it works? RPi forum thread gives a general idea that applies in my case:

1. Divide the 3×4 matrix as columns and rows. 4 GPIO pins as rows are pulled-up with 10k resistors and initialized as input.
2. Other 3 GPIO pins as columns are initialized as output low.
3. First loop will scan for one pressed key being read as one of the rows pulled-low
4. After the loop breaks, all columns are set as input, then the row pin found in the loop is set as output-high
5. Second loop will scan for column being pulled-high by that row pin. Between both loops, it is assumed that the key press is still in effect. In reality, normal human act of pressing this key elapses long enough for the software to run the scans in two loops.
6. Bingo! The code reads row-column combination of the pressed key.

Example of the code’s output by calling from shell:

$ while true; do /usr/local/bin/matrixQPi.py -i; done
3
6
9
8
#
*

(In the above example ^C will throw Python KeyboardInterrupt messages before breaking the bash loop)

GPIO pin logic state (meaning voltage) are both programmable and driven by physical-connection. I choose wiringPi for practical reasons: availability of its Python wrapper and its simple syntax (glancing it at first sight). WiringPi has an option of using its own pin numbering to address it in the code instead of the original GPIO numbering (there are board revisions to watch for in some cases of usage, not mine). Every pin can be initialized as input or output.

Updated: Wiring-Pi Python is deprecated and moving to 2.x version that supports I/O expander. However, you can still find this combination of commits that will work and build without error message:

- main module: WiringPi-Python@9c77bde
- submodule: WiringPi@89bbe97

(Check how to build on my README)

Using an analog multitester, here are behavioral findings:

• GPIO pin voltage swings to high logic after reboot (the meter’s needle is rocking for a second or so). It is in low voltage afterward until being programmed or physically pulled-up.
• Two pins, wiringPi pin 8 and 9, remain in high logic voltage after reboot. They are SDA0 and SCL0 to be used as an I²C, however they can be used to read button as well. (eLinux Wiki: “there are 1.8 k pulls up resistors on the board for these pins”)
• I did short the high logic to ground accidentally, it invoked reboot. (I don’t know how many times or how long of this will brick the Raspberry Pi)

Prior to trying the push-button switch, I didn’t have proper circuitry and working with wires as probes, prone to accident that was. I found an advice to insulate the +5V pin voltage so I could worry less.

Insulate the +5V pin of Raspberry Pi

A 10k pull-up resistor and a button are enough to test the following. Go to interactive Python shell and run line by line until the button push is read as low logic (GPIO7 in this example):

# python
Python 2.7.3rc2 (default, May  6 2012, 20:02:25) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import wiringpi
>>> import time
>>> INPUT=0
>>> OUTPUT=1
>>> HIGH=1
>>> LOW=0
>>> SETUP=wiringpi.wiringPiSetup()
>>> print SETUP
0
>>> wiringpi.pinMode(7,INPUT)
>>> RESULT=wiringpi.digitalRead(7)
>>> print RESULT
1
>>> RESULT=wiringpi.digitalRead(7)
>>> print RESULT
1
>>> RESULT=wiringpi.digitalRead(7)
>>> print RESULT
0
>>>

These basics convince me to go ahead with my goal of having a membrane (matrix) keypad as input for Raspberry Pi to run some script (see next post).

(Check also RPi common USB problems post)

1. download source file (that is ffmpeg in this example)
2. retrieve development library/build dependecies
3. place header files and dependency libraries into toolchain path
4. configure with toolchain
5. make and create .deb installer package with checkinstall

cross-compile: Illustration of what's done on ARM-board and what's done on desktop

Prior to anything, install Linaro toolchain by adding their repo first:

$ sudo add-apt-repository ppa:linaro-maintainers/toolchain
$ sudo apt-get install gcc-arm-linux-gnueabi

A complete guide on how to compile ffmpeg can be found in their wiki. It applies generally. So, I’ll just skip to following configure:

[ubuntu-desktop]$ ./configure --cross-prefix=arm-linux-gnueabi- --enable-cross-compile --target-os=linux --arch=arm  --cpu=cortex-a8 --extra-cflags='-march=armv7-a -mfpu=neon -mfloat-abi=softfp' --enable-libtheora

As an example, the above configure requires Theora libraries. libtheora-dev itself requires libogg-dev. Use apt-get on the board to get them from Ubuntu ARM repository. Get the .deb files from /var/cache/apt/archives/ of the ARM-board to your desktop and extract them as root:

[root@ubuntu-desktop]$ dpkg -x libogg-dev_1.2.2~dfsg-1ubuntu1_armel.deb /usr/local/cross-compile
[root@ubuntu-desktop]$ dpkg -x libtheora-dev_1.1.1+dfsg.1-3_armel.deb /usr/local/cross-compile

The above /usr/local/cross-compile is just a temporary directory. The toolchain needs header files and dependency libraries to be placed inside /usr/arm-linux-gnueabi/include/ and /usr/arm-linux-gnueabi/lib/ consecutively. In my case I created the following link for header’s include:

[root@ubuntu-desktop]$ cd /usr/arm-linux-gnueabi/include
[root@ubuntu-desktop]$ ln -s /usr/local/cross-compile/usr/include/theora
[root@ubuntu-desktop]$ ln -s /usr/local/cross-compile/usr/include/ogg

and copied these from /usr/local/cross-compile/usr/lib:

libogg.a
libogg.so
libtheora.a
libtheoradec.a
libtheoradec.la
libtheoradec.so
libtheoraenc.a
libtheoraenc.la
libtheoraenc.so
libtheora.la
libtheora.so

After successful configure, run make (this will take some time), and then pack the whole working directory to the ARM-board. Instead of doing make install in the ARM-board, use checkinstall to install and create the .deb package. This way we can make use of dpkg to uninstall or re-install in the manner of package manager.

[root@beagleboard]$ sudo checkinstall --pkgname=ffmpeg --pkgversion="5:$(./version.sh)" \
     --backup=no --deldoc=yes --default

Please mind that even if we put “--install=no“, the above checkinstall will overwrite all ffmpeg binaries in /usr/local/bin/. This is the reason why I move the process to the ARM-board: to keep other running version on my desktop stays as it is.

Check the test of streaming from our cross-compiled ffmpeg in previous post.

A Little History

When first attempt to configure exited with ERROR: libtheora not found, I found the following test failed in the output log:

$ arm-linux-gnueabi-gcc -D_ISOC99_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_POSIX_C_SOURCE=200112 -D_XOPEN_SOURCE=600 -march=armv7-a -mfpu=neon -mfloat-abi=softfp -mcpu=cortex-a8 -std=c99 -fomit-frame-pointer -marm -pthread -E -o /tmp/ffconf.UdYctJgq.o /tmp/ffconf.mwXpf9Tl.c
...
/usr/lib/gcc/arm-linux-gnueabi/4.6.1/../../../../arm-linux-gnueabi/bin/ld: cannot find -ltheoraenc
/usr/lib/gcc/arm-linux-gnueabi/4.6.1/../../../../arm-linux-gnueabi/bin/ld: cannot find -ltheoradec
/usr/lib/gcc/arm-linux-gnueabi/4.6.1/../../../../arm-linux-gnueabi/bin/ld: cannot find -logg

What was inside the above temporary file (/tmp/ffconf.mwXpf9Tl.c):

1
2

#include <theora/theoraenc.h>
int x;

You can run that test to check whether all dependencies are met.

$ ffmpeg -i UNIQLO_MIXPLAY.flv -v 0 -vcodec mpeg4 -f mpegts udp:192.168.1.19:1234

ffmpeg streaming from BeagleBoard played by VLC on the desktop

Although it’s a break-dance video, it plays uninterrupted, without any break.

Here is how my desktop see the stream with VLC player:

VLC setup (on the desktop) to play the UDP stream

With less documentation, a more complex setup with integration to Wowza Media Server is available on my github.

1. independent SNMP data polling by individual NMS proxy (embedded system)
2. intermittent connection between main/master NMS server with its proxy

Of course we’ll do them in implementation context of Zabbix Server as main/master NMS server and our BeagleBoard xM as its proxy.

Zabbix Proxy - distributed NMS over embedded Linux

In the technical details, I already followed pretty much the same steps showed in installation wiki (with the exception that I didn’t put the process into startup init.d script as I would start it manually as needed).

The first test is to make our Ubuntu ARM board accepts configuration done in Zabbix Server web-GUI, retrieves SNMP data to it, and sends collected data to server. Mode passive is chosen so that the server will be the one initiating contact with the board. After enabling a certain SNMP host, i.e. host-aaa, as monitored-by-proxy, the Zabbix Server log will show:

15677:20120601:022022.960 Sending configuration data to proxy 'beaglexm1'. Datalen 9205

Our board, ‘beaglexm1‘, will start to poll SNMP data from host-xxx to then store it in designated sqlite database.

975:20120531:212024.098 proxy #8 started [poller #5]
984:20120531:212024.128 proxy #17 started [discoverer #1]
968:20120531:212025.423 Received configuration data from server. Datalen 9205
971:20120531:212030.388 Enabling SNMP host [host-aaa]

From time to time as configured, our board will also run housekeeping to keep our board database file in small size.

1081:20120531:224513.922 Executing housekeeper
1081:20120531:224514.505 Deleted 4885 records from history [0.453765 seconds]

Under normal connection, in main Zabbix Server we’ll start seeing the collected data from host-aaa the same way as when the SNMP data is polled by the server directly. To emulate disruption to the connectivity, I choose to use simple iptables drop packet. This time the server will no longer seeing new data.

After the drop block is released, the server we’ll start seeing new data as well as data from the period of intermittent connection. It will take some time before all of that being sent to the server.

In general that proves it. I collected 44 OID values from the host-aaa every 30 seconds and in this case BeagleBoard utilization was very low as this wasn’t a stress test to see its performance.