PyZine
 


Article Finder
People
Issue 7 - Revision 6  /   February 27, 2005 


 
  Py Links:
Latest Issue
Issue 08
Issue 07
Issue 06
Issue 05
Issue 04
Issue 02
Issue 01
 
 
Downloads
     
  Articles:
Throughout the quarter we cover topics of interest to Python developers.

COM & Python

Python on .NET

Python at Both Ends of the Web

GUI Testing Approach

Simulating with SimPy

Docutils

Mobile Collection System

 
 
 
     


Using Python to Create a Mobile Data Collection System

- - - - - - - - - - - -

By Peter Kropf  | February 27, 2005

print

Introduction

Python is an extremely versatile programming language. In fact, Python is so versatile that it was an ideal choice to control a (very) portable computer system strapped to a motorcyle. A custom-built Mini-ITX mobile data collection system collected temperature, latitude, longitude, elevation, and video images during a recent transcontinental motorcyle ride. Let's take a look at the system and see how Python was used to manage all of it.

Building a mobile computer

When taking a motorcycle trip across the United States, a die-hard geek has to ask the question, "How can one use technology to assist traveling?" The answer is obvious: use different technologies to record the places visited, including images and each destination's temperature, location, elevation.

But to do all that recording, one must first build a computer that's small, portable, and reliable.

To start, take a Mini-ITX motherboard and case, add some memory and a really big hard disk. Add a temperature sensor, a video capture card, a video camera, and a GPS to round out the hardware. Then there's the issue of power. Since the system was going to be powered from the electrical system of a motorcycle, an ITPS Power Sequencer was chosen to control boot and shutdown sequences on the system. A schematic of the entire system is shown in Figure One.

SystemHardwareOverview.png

Figure One: A high-level schematic of the data collection system

Since images were going to be collected, a 200 GB hard disk was chosen. Unfortunately, this caused a problem with the Mini-ITX case that was purchased: the case was designed to use 2.5" laptop hard disks. The largest 2.5" disk was 80 GB, which was both too small and much more expensive than a 3.5" hard disk. So a compromise was made and the top of the case wasn't used. This allowed the hard disk to be mounted where the thin CD-ROM drive was supposed to go. (See Figure Two.)

HardDriveMounting.png

Figure Two: A picture of the hard drive mounted in the top of the case

The sensors and controllers used in the system were manufactured by Dallas Semiconductor (http://www.dalsemi.com/). Based on the company's 1-Wire networking technology, the sensors and controllers use a single wire (plus a ground) for both communication and power transmission. Moreover, the components can be fed by a single bus master, and are uniquely addressable (a real advantage, as you'll see shortly). Better yet, the devices can be controlled from a standard computer system running Debian Linux. (If you're a hardware buff, take a look at the Dallas Semiconductor web site for all of the fine details.)

The Dallas Semiconductor TAI8520 1-Wire temperature sensor and DS9490R USB adapter were a perfect match to collect temperature data. Since the system was going to be mounted inside of a sealed side case on the motorcycle, two temperature sensors were used. One sensor was mounted on the frame of the motorcycle and exposed to the outside air. The second was mounted inside the case to monitor the internal temperature of the system in an effort to prevent a complete meltdown of the computer due to heat. The internal sensor is pictured in Figure Three.

TemperatureSensor.png

Figure Three: The system temperature sensor

An ATI TV Wonder VE NTSC PCI video capture card was installed in the system to capture video images. The TV Wonder card is well supported by the Video4Linux project (http://www.video4linux.net/).

The choice of video cameras quickly narrowed down to the Sony HI RES CCD from RF Concepts (http://www.rfconcepts.co.uk/sony_colour_ccd.htm). This is a small, waterproof, so-called "bullet" camera that produces great video images. The fact that it is also fairly small made mounting problems almost non-existent. You can see the camera mounted at the front of the motorcyle in Figure Four.

MountedVideoCamera.png

Figure Four: The Sony HI RES CCD "bullet" camera installed in the front of the motorcyle

The Garmin GPS V (http://www.garmin.com/) is an older GPS model but one that works very well. It can be configured to provide location and elevation data via a standard RS-232 serial line. Since the Epia-M motherboard provides one serial port, connecting the GPS V was pretty simple.

The ITPS Power Sequencer (http://www.logicsupply.com/product_info.php/products_id/162) is a great little power controller. It takes 12V switched power and 12V unswitched power from a motorcycle (or car) and provides a logical computer power button connection and power for the computer. The logical power button is connected to the pins on the motherboard where the power switch from computer case would normally go, and the power for the computer is connected to the normal computer power plug. On the C-138 case, this connects the ITPS output power to an internal DC power board which is normally connected to a power brick.

With these connections made, the system is ready to be powered on. When the switched power is turned on via the vehicle's key, the ITPS waits for a bit then turns on power to the computer and logically presses the power button. When the switched power is turned off, the ITPS waits a bit and then logically presses the power button again. After 45 seconds, power to the computer is turned off. So, the system has to be configured to shutdown when the power button is pressed and shutdown should to take less than 45 seconds to complete.

The final system is shown mounted and ready to go in Figure Five.

CaseInSaddleBag.png

Figure Five: The computer in the saddle bag of the motorcycle

Operating Systems Choices

The choice of operating system for this project was pretty simple: Linux. Since cost was an issue, Windows and all of its variants were quickly discarded. Debian Linux was the final choice because it's easy to install, contains many of the leading edge packages that would be needed, and best of all, it's free.

Before the operating system was installed, a few things had to be changed in the BIOS configuration. Specifically, accessories that were not going to be used were turned off. This included the floppy disk controller and the printer port. Additionally, the BIOS was configured to ignore all errors. This would allow the system to boot with no keyboard connected.

Installing the operating system was pretty simple: just boot from the Debian installer CD and select the packages that are needed. Like many Linux installations, this proceeded without any problems. The system was installed pretty quickly and all of the attached hardware was detected.

Power Problems

The initial build of the system and the installation of the operating system was done with the hardware off of the motorcycle and on a workbench. (There's a lot more space on a workbench than there is in the side luggage case on a motorcycle.)

While the complete build on the workbench went smoothly, the system failed to power up when initially attached to the motorcycle. A couple of LEDs were on, but there was no BIOS POST (Power On Self Test), and the hard disk didn't feel like it was spinning. When doing the work on the workbench, a standard 90 watt power brick was used. But when connected to the motorcycle, the ITPS was used.

Poking around a bit, it looked like the initial startup power requirements of the hard disk exceeded the abilities of the ITPS. So, a simple single pole 12V relay was inserted into the circuit. The ITPS main power line was used to drive the 12V relay. When the relay was activated, it would provide power to the system. Once this was done, the system powered on without any further problems.

Data Collection

The chore of collecting data can be achieved through several different means. Python was chosen for several reasons:

1. The "batteries included" concept of Python meets many of the base needs of the project.

2. The availability of third party libraries to communicate and control sensors simplifies coding.

  1. Debugging Python code is fairly easy.
  2. And, to borrow a statement from Bruce Eckel, Python fits your brain.

There are three types of sensors being used in this project: temperature, location, and video. For each of these, there are three aspects to consider:

  1. Actions that need to happen at system startup to support the sensor
  2. How will the data be collected and stored
  3. Actions that need to happen at system shutdown to clean up

We'll take a look at each of the sensors and the three aspects to support them.

Collecting Temperature Data

The temperature sensor data is available via a 1-Wire module connected to the system via the USB bus. Writing Python (or any other language) code to directly communicate with the module over USB is a non-trivial task. But in this case, we didn't have to. The OWFS (http://owfs.sourceforge.net/) project provides access to 1-Wire devices via the filesystem, thereby greatly simplifying the code that has to be written to communicate with 1-Wire devices.

At system startup, the OWFS file system has to be mounted. Because there isn't an easy way to specify OWFS file systems in /etc/fstab, an init script is used to mount the file system.

 1 #! /bin/sh
 2 # $Id: owfs.init 18 2004-07-21 20:24:25Z peter $
 3 
 4 PATH=/opt/owfs/bin:/bin:/usr/bin:/sbin:/usr/sbin
 5 DAEMON=/opt/owfs/bin/owfs
 6 PIDFILE=/var/run/owfs.pid
 7 
 8 # Arguments to owfs
 9 #
10 MOUNT_POINT="/1-wire"
11 ARGS="-F -u $MOUNT_POINT"
12 
13 test -x $DAEMON || exit 0
14 
15 case "$1" in
16   start)
17     echo -n "Starting owfs"
18     start-stop-daemon --start --quiet --pidfile $PIDFILE \
        --exec $DAEMON --$ARGS
19     echo "."
20     ;;
21   stop)
22     echo -n "Stopping owfs"
23     umount $MOUNT_POINT
24     echo "."
25     ;;
26   restart)
27     sh $0 stop
28     sh $0 start
29     ;;
30   *)
31     echo "Usage: /etc/init.d/owfs {start|stop|restart}"
32     exit 1
33     ;;
34 esac
35 
36 exit 0

The owfs.init script is fairly simple and typical of a daemon startup script. When the start parameter (line 16) is specified, it starts the OWFS daemon telling it to mount the 1-Wire sensors under /1-wire.

A difference to other init scripts is when the stop parameter (line 21) is specified. Instead of stopping the daemon itself, the /1-wire file system is unmounted. This cleans up the kernel file system entries and such, as well as stopping the OWFS daemon process. The restart parameter (line 26) is just a simple way of issuing a stop followed by a start.

When the OWFS daemon mounts a file system, it queries the USB bus to find any 1-Wire sensors. For each of the sensors found, it queries the sensor to obtain it's unique address. With 1-Wire sensors, each sensor ever created has an address that Dallas Semiconductors insist is globally unique. A directory listing of /1-wire shows...

10.B7B64D000800/ 10.54BA4D000800/

... which are the addresses for the temperature sensors used on this project. Within each of those two directories, there are a number of files that provide access to all the details of the sensors. Of particular interest is the temperature file. Opening and reading from this gives the temperature as a string of a floating point number.

With 1-Wire sensors and OWFS, gaining access to the data is very easy. Just open the appropriate file and read the data. (At least that's the case for the temperature sensors. If time had permitted for this project, a humidity sensor would have been added. However, reading the humidity data isn't as simple as reading the temperature. And since this project had to be finished before the road trip started, the humidity sensor was dropped.)

The following script, one_wire_sensors.py, monitors the temperature.

  1 #! /usr/bin/env python
  2 
  3 # $Id: one_wire_sensors.py 23 2004-07-22 04:35:36Z peter $
  4 #
  5 # need to check for /1-wire mounted but Vendor=04fa is not
  6 # in the usb usb devices file. this can occur when the usb
  7 # dongle is no longer connected.
  8 #
  9 # check that /1-wire is mounted
 10 # if not, check that Vendor=04fa is in the usb devices file.
 11 # if it is, then kick /etc/init.d/owfs to start up again
 12 # if not, then ?
 13 
 14 
 15 import os
 16 import syslog
 17 import csv
 18 import time
 19 import util
 20 
 21 
 22 Debug           = True
 23 OWUSBVendor     = '04fa'
 24 USBDevices      = '/proc/bus/usb/devices'
 25 MountFile       = '/proc/mounts'
 26 OWUSBMount      = '/1-wire'
 27 MountCommand    = '/etc/init.d/owfs.init start'
 28 UnmountCommand  = '/etc/init.d/owfs.init stop'
 29 SensorLogName   = 'sensor.log'
 30 Sensors         = {
 31     'Temperature1' : os.path.join( OWUSBMount, '10.B7B64D000800' ),
 32     'Temperature2' : os.path.join( OWUSBMount, '10.54BA4D000800' ),
 33     #'Humidity1'   : os.path.join( OWUSBMount, '12.386623000000' ),
 34     #'Humidity2'   : os.path.join( OWUSBMount, '12.3B6F23000000' ),
 35    }
 36 Messages        = {
 37     'MountedNoPresence' :
 38     '1-wire mounted but no usb presence, unmounting 1-wire',
 39 
 40     'UnmountedPresence' :
 41     '1-wire not mounted but usb presence, mounting 1-wire',
 42 
 43     'UnmountedNoPresence' :
 44     '1-wire not mounted and no usb presence, nothing can be done',
 45 
 46     'MountedNoSensors' :
 47     '1-wire mounted and usb presense but no sensors, remounting 1-wire',
 48     }
 49 
 50 
 51 def log_message( message ):
 52     if Debug:
 53         print message
 54     syslog.syslog( syslog.LOG_DAEMON|syslog.LOG_WARNING, message )
 55 
 56 
 57 def check_mount( mount_point ):
 58     known_mounts = open( MountFile, 'r' ).readlines()
 59     for mount in known_mounts:
 60         if mount_point in mount:
 61             return True
 62     return False
 63 
 64 
 65 def check_usb_vendor( vendor ):
 66     usb_lines = open( USBDevices, 'r' ).readlines( )
 67     vendor = 'Vendor=' + vendor
 68     for line in usb_lines:
 69         if vendor in line:
 70             return True
 71     return False
 72 
 73 
 74 def check_sensors( ):
 75     for sensor in Sensors:
 76         if not os.path.isdir( Sensors[ sensor ] ):
 77             return False
 78     return True
 79 
 80 
 81 def read_temperature( ):
 82     path = os.path.join( Sensors[ 'Temperature1' ], 'temperature' )
 83     temperature1 = open( path, 'r' ).readlines( )
 84     path = os.path.join( Sensors[ 'Temperature1' ], 'temperature' )
 85     temperature2 = open( path, 'r' ).readlines( )
 86     return ( float( temperature1[ 0 ] ), float( temperature2[ 0 ] ) )
 87 
 88 
 89 def log_sensors( ):
 90     current_time = time.time( )
 91     path = util.log_path( '', SensorLogName )
 92     temperature1, temperature2 = read_temperature( )
 93     writer = csv.writer( file( path, 'a' ) )
 94     writer.writerow( ( current_time, temperature1, temperature2 ) )
 95 
 96 
 97 def one_wire( ):
 98     mounted      = check_mount( OWUSBMount )
 99     usb_presence = check_usb_vendor( OWUSBVendor )
100     sensors      = check_sensors( )
101 
102     if mounted:
103         if not usb_presence:
104             log_message( Messages[ 'MountedNoPresence' ] )
105             os.system( UnmountCommand )
106         elif not sensors:
107             log_message( Messages[ 'MountedNoSensors' ] )
108             os.system( UnmountCommand )
109             os.system( MountCommand )
110         elif Debug:
111             print 'all is right with the world'
112     elif usb_presence:
113         log_message( Messages[ 'UnmountedPresence' ] )
114         os.system( MountCommand )
115     else:
116         log_message( Messages[ 'UnmountedNoPresence' ] )
117 
118 
119 if __name__ == '__main__':
120     one_wire( )
121     if check_mount( OWUSBMount ):
122         log_sensors( )

A crontab entry was used to run one_wire_sensors.py every 5 minutes to collect the temperature data.

One of the more annoying issues discovered once this portion of the system was operational was that the sensor entries would disappear from the /1-wire directory. A bit of physical world debugging showed that the USB adapter used to connect the 1-Wire sensors to the system would jiggle in the socket. This resulted in a USB disconnect and reconnect happening in the kernel. Once the USB disconnect happened, the OWFS daemon would lose it's connection to the sensors and would drop the entries.

With the system being mounted on a motorcycle and subject to bounces from the road surface, this would be a problem. To get around this, the one_wire() function (line 97) is used to verify that the /1-wire directory is mounted and the sensors are accessible.

Collecting Location Data

A GPS is an amazing device. Essentially, it receives faint time signals from a constellation of 24 satellites orbiting the earth at around 12,000 miles and compares those values to a known time value to compute how long it took the signals from different satellites to travel to the GPS. Based on how long the signals took, the GPS receiver is able to calculate the near-exact location of the receiver on the earth.

At system startup, the gps.init script (shown immediately below) starts a process running gps.py to collect the data.

 1 #! /bin/sh
 2 # $Id: gps.init 18 2004-07-21 20:24:25Z peter $
 3 
 4 PATH=/bin:/usr/bin:/sbin:/usr/sbin
 5 DAEMON=/root/ltodyssey/bin/gps.py
 6 PIDFILE=/var/run/gps.pid
 7 
 8 ARGS=""
 9 
10 test -x $DAEMON || exit 0
11 
12 case "$1" in
13   start)
14     echo -n "Starting GPS capture - gps.py"
15     start-stop-daemon --start --pidfile $PIDFILE --make-pidfile \
        --background --exec $DAEMON -- $ARGS
16     echo "."
17     ;;
18   stop)
19     echo -n "Stopping GPS capture - gps.py"
20     start-stop-daemon --stop --pidfile $PIDFILE
21     echo "."
22     ;;
23   restart)
24     sh $0 stop
25     sh $0 start
26     ;;
27   *)
28     echo "Usage: /etc/init.d/gps.init {start|stop|restart}"
29     exit 1
30     ;;
31 esac
32 
33 exit 0

The gps.init script is again simple and is very similar to the owfs.init script. When start is passed to it, a daemon process is started to execute the gps.py code. When stop is passed to it, it stops the daemon process. As with owfs.init, the restart command is just a simple way of issuing a stop followed by a start.

For this project, a Garmin GPS V was used. The GPS V is connected to a computer via an old fashion RS-232 serial port. Now, writing RS-232 serial port code is annoying. But again, we don't have to, thanks to the wondrous PyGarmin project (http://pygarmin.sourceforge.net/). PyGarmin provides a set of Python classes that implement the protocol used by Garmin GPS receivers. (This one of the great features of using Python: modules like this exist and make life much easier for developers.)

 1 #! /usr/bin/env python
 2 # $Id: gps.py 23 2004-07-22 04:35:36Z peter $
 3 
 4 
 5 import os
 6 import sys
 7 import time
 8 import csv
 9 from pygarmin import garmin
10 import util
11 
12 
13 GPSDevice      = '/dev/ttyS0'
14 GPSLogName     = 'gps.log'
15 GPSCollectRate = 3  # number of seconds between snapshots
16 
17 
18 def connect( device ):
19     phys = garmin.UnixSerialLink( device )
20     gps = garmin.Garmin( phys )
21     gps.pvtLink.dataOn( )
22     return gps
23 
24 
25 if __name__ == '__main__':
26     while 1:
27         try:
28             gps = connect( GPSDevice )
29             while 1:
30                 start_time = time.time( )
31                 position = gps.pvtLink.getData( )
32                 fullname = util.log_path( str( position.wn_days ), 
                                   GPSLogName )
33                 writer = csv.writer( file( fullname, 'a' ) )
34                 writer.writerow( ( start_time,
35                                    position.alt,
36                                    position.east,
37                                    position.epe,
38                                    position.eph,
39                                    position.epv,
40                                    position.fix,
41                                    position.leap_secs,
42                                    position.msl_height,
43                                    position.north,
44                                    position.rlat,
45                                    position.rlon,
46                                    position.tow,
47                                    position.up,
48                                    position.wn_days ) )
49 
50                 end_time = time.time( )
51                 time_difference = end_time - start_time
52                 if time_difference < GPSCollectRate:
53                     time.sleep( GPSCollectRate - time_difference )
54         except garmin.LinkException, e:
55             print 'received a LinkException:', e
56             print 'trying to reestablish the GPS connection'
57             time.sleep( GPSCollectRate )

The basic logic behind the gps.py code is pretty straight forward: connect to the GPS, read the current location data, write that data out to a comma separated file, and sleep for a bit before starting over again. The connect() function (line 18) connects to the GPS attached to the serial port specified by GPSDevice (line 13) and tells it to start collecting data (line 21.)

Once a connection is established, the code loops (line 29) through obtaining the current location data (line 31), writing that data out to a CSV file (line 34) and waiting for a period of time before reading the data again.

The whole connect, collect data, and save it loop is defined within a try/except block (lines 27-54). When a garmin.LinkException is caught, it indicates that there was a problem in communicating with the GPS. (From experience, this was typically a result of user error. While riding, I would sometimes press the power button instead of something else. This would mostly occur when I would try to manipulate the GPS while wearing riding gloves. The buttons on the GPS V are pretty small and leather gloves made it very difficult to press individual buttons.) So, the try/except block within a while loop (line 26) ensured that the process of collecting data would continue or at least attempt to continue in the event of serial line communication problems.

In a perhaps vain attempt at limiting possible loss of data, the CSV data file is changed every week. This is done by using the wn_days field (line 32) from the collected location data. The wn_days field provides the number of days that have occurred from UTC December 31, 1989 to the beginning of the current week. That gives the current GPS week number. By using the current GPS week number in the log file path, at most a weeks' worth of data would be lost. Or at least, that was the thought when I was originally putting this system together. In hind sight, I'm not sure that it provided all that much protection.

One very glaring mistake was the use of a timer within the code. The timer caused the code to wait for 3 seconds between reading GPS location data. The original thought was that the GPS location data was to be related to the video images captured and should thus be at the same frequency as the video image capture. A side effect of this is that using the GPS data to plot path traveled maps would be problematic as the resolution of the map was increased. Traveling at 75 MPH, the distance traveled in 3 seconds is 330 feet. This 330 feet compounds the inherent GPS location accuracy. Once more work is done on post processing the data, a better idea on the impact of this choice can be made.

Collecting Video Images

Collecting video images along the route traveled was one of the goals for the project. One way of doing this would be to use a standard video camera and change tapes every hour. Riding an average of six hours a day for five weeks would result in some 200+ tapes. Considering that the cost of tapes alone would break the budget for this project and that the time needed afterward to edit would be fairly lengthly, a decision was made to work out how to record directly to disk.

The basic requirements for the camera were color video, waterproof or easily waterproofed, small, powered by a 12V DC source, and not that expensive. The first camera that was tried was the Apple iSight since I already owned one. Unfortunately, it ran into a waterproofing problem. The iSight generates a substantial amount of heat while operating, and I was concerned about putting it in a waterproof enclosure. Not knowing how much heat the camera can tolerate, I decided to look for a different camera.

Some online research yielded a number of different cameras know as helmet cameras or bullet cameras. Of the ones that are available, I settled on a Sony High Res CCD camera from RF Concepts. This is a really small camera that takes pretty good pictures, as shown in Figure Six.

VideoImageSample.jpg

Figure Six: A sample of the image quality of the Sony High Res CCD camera

Capturing the video was a big question for me and I was glad to find that the Video4Linux project provides great software for this type of work. I was also concerned about running out of disk space trying to capture 30 frames per second riding up to eight hours a day for four or five weeks. So, I decided to take periodic snapshots. I settled on capturing a video image once every 3 seconds. I also struggled for a bit regarding how to capture the video itself. I wanted to do all the work in Python but realized that there wasn't enough time before I leaving to allow me to write the code that would be needed. Instead, I decided to use Python to take snapshots using the v4lctl program that's part of the Video4Linux project. The result is the v4l.py script.

 1 #! /usr/bin/env python
 2 # $Id: v4l.py 25 2004-07-22 19:48:22Z peter $
 3 
 4 
 5 import os
 6 import sys
 7 import time
 8 import util
 9 
10 
11 V4LDevice    = '/dev/video0'
12 V4LCTL       = '/usr/bin/v4lctl -c ' + V4LDevice
13 SnapshotRate = 3  # number of seconds between snapshots
14 
15 
16 def command( cmd ):
17     cmd = V4LCTL + ' ' + cmd
18     ret = os.system( cmd )
19     
20 
21 def setup( ):
22     command( ' setnorm  NTSC' )
23     command( ' setinput Composite1' )
24     command( ' capture  grabdisplay' )
25     command( ' color    "50%"' )
26     command( ' hue      "50%"' )
27     command( ' bright   "50%"' )
28     command( ' contrast "50%"' )
29 
30 
31 def snapshot( file ):
32     command( ' snap jpeg full ' + file )
33 
34 
35 def collect_snapshots( ):
36     while 1:
37         start_time = time.time( )
38 
39         year, month, day, \
40             hour, minute, seconds,
41             weekday, yearday, dst = time.localtime( start_time )
42 
43         snapshot_name = '%d_%02d_%02d_%02d_%02d_%02d.jpg' % (
44             year, month, day, hour, minute, seconds )
45         fullname = util.log_path( os.path.join( '%d' % year,
46             '%02d' % month, '%02d' % day,   '%02d' % hour,
47             '%02d' % minute ), snapshot_name )
48         snapshot( fullname )
49 
50         end_time = time.time( )
51         time_difference = end_time - start_time
52         if time_difference < SnapshotRate:
53             time.sleep( SnapshotRate - time_difference )
54 
55 
56 if __name__ == '__main__':
57     setup( )
58     if len( sys.argv ) == 1:
59         collect_snapshots( )
60     else:
61         snapshot( sys.argv[ 1 ] )

The v4l.py code sets up the Video4Linux subsystem to find the camera and take NTSC type video. It then creates a snapshot image that uses a time stamp in the name to ensure uniqueness and order to the images. It also has the ability to take an individual snapshot which helped with debugging the hardware.

Post Processing Images

The plan was to generate videos while on the road and post them to a web site. In reality, that didn't work out all that well. The first videos that I created were annoying because the software I was using insisted on inserting the .jpg images into the video on what seemed like 1 second boundaries. The result was that when watching the video, an image is shown for 1 second before the next one. That's not quite the effect that I wanted to achieve.

After the trip was finished, I did a bit more online research on creating videos from images and found that mpeg2enc does exactly what I wanted. At least, it was supposed to do what I wanted. I created a daily video using:

1 find /space/ltodyssey/sensors/space0/2004/ -name \*.jpg | \
2   xargs -n1 jpegtopnm | ppmtoy4m | mpeg2enc -o ltodyssey.mpg

While that looks like it should work and a video was produced, playing it was odd. It seemed that the images were bouncing around in time. What I realized is that the output of find is not necessarily in alphabetical order.

1 find /space/ltodyssey/sensors/space0/2004/ -name \*.jpg | \
2   | sort | xargs -n1 jpegtopnm | ppmtoy4m \
3   | mpeg2enc -o ltodyssey.mpg

Sorting the list of .jpg files fixed that problem.

A single video, covering all the images from the trip at 640 by 480 resolution produces a video that's 1.3 GB. That's a wee bit too much to be downloaded from the Internet. So I settled on creating a video for each day. These are a bit more manageable at around 40 MB.

In hind sight, I should have increased the frequency to 1 per second or even 3 per second. All told, 147,945 images were captured taking up 7.6 GB of disk space, which was a fraction of the 200 GB of space that was available. One image every 3 seconds makes the generated videos a bit more choppy than they could have been.

I've also thought about creating a better viewer that incorporates the images, GPS location data, mapping sources, and the temperature to allow for watching a map for location and the images at the same time. But I'm not certain when I'll be able to get to that project.

Summary

Rushing to get all everything together and working before heading out on the road was a bit stressful, but I'm very glad I went through the effort. This was a great mix of tinkering with hardware and writing Python code. If Python wasn't available, I'm not sure that I could have finished in time.

There's a travel journal at http://ltodyssey.org where you'll find thoughts, maps, images, and videos. If you stop by, leave me a note and let me know what you think.

And what was I riding? Here's the cycle fully-loaded.

K1200LT.png


Peter Kropf

shim
shim

 Py is committed to bringing you great Python Articles.

shim
shim


Home   Subscribe   Migration FAQ   Contact PyZine   Write for PyZine   ZopeMag   opensourcexperts.com  

Reproduction of material from any of PyZine's pages without prior written permission is strictly prohibited. Copyright 2003 - 2005 PyZine Zope/Plone hosting by Nidelven IT