Magellan Technology is an R&D company working on a range of RFID products, ranging from chip design, through tag and reader manufacture. Some of the readers are simple, such as a small desktop reader, where others are highly complex conveyor belt readers.

This article discusses how we used Python for many aspects of our software development, both on and off the readers. Also, we pass on our views of the current controversy over privacy and RFID.

There have been many articles published recentlydiscussing the privacy implications of RFID. Most of them are overhyped and inaccurate, to such a degree that one could conclude that some articles are FUD from other identification industries, which see RFID as a threat.

Many articles refer to trials at high-profile companies, which are announced with a great deal of fanfare, but later cancelled at a very early stage with the numbers of tags involved being nothing more than marketing hype. While many of these trials have been cancelled, they are still being referred to in articles as if they are a reality.

The RFID industry as a whole is extremely fragmented, with many different frequencies, tag types and low-level protocols, which makes large-scale tag reading in a public space near to impossible. Even with a single, widely used tag, each market stores data on the tag in its own format, making multi-supplier analysis of a user through RFID difficult.

It would be a software nightmare to try and process all the different physical and format differences between all the tags in current usage in order to gather general customer information.

There has been widespread discussion of using EPC (electronic product code) as a cross industry method, but this is still in its infancy and has numerous problems with the current technology.

In many of these products, there is a significant gap between the marketing and the reality of the technology.

There is also the issue of read range. Apart from powered tags, which are commonly used in cars for freeway tolls, RFID tags must get enough powering field from the reader to make them function. Most tags have a range limit of under a meter, because human RF exposure laws limit the power of the transmitters.

So the idea of scanning people as they walk along the street is a little fanciful. There are no doubt scenarios where it would be possible, but there are very definite technical limits as to range.

There are some tags with a longer range, but they usually are very simple devices with a serial number only and these devices are read-only, not writeable.

Even with all the technical limitations of current technology for Big Brother scenarios (as in Orwell, not the TV series), it is of course possible that these limits will be reduced in the future to the point that some potential for abuse would arise.

One example that is frequently given is: you buy an item, such as a piece of clothing, which has a sewn-in RFID tag, and the tag can subsequently be used to identify you. However, reading the tag typically gives you only an identification of what kind of item the tag is in, and perhaps a unique serial number as well (though many systems do not have this, so every item of the same kind has exactly the same data).

The obvious question here is: what is there in the tag that can be associated with a given customer? If a customer is really worried, he should pay cash. Then there is no traceable trail. But even if he does pay by credit card, let us imagine what would have to happen for Big Brother to track him and invade his privacy:

The retailer would have to set up a database of all sales and record credit card and other customer details. Other retailers would have to do the same for the products they sell, using the same format, and preferably all of this information would then be stored in a huge central system run by Big Brother.

What retailer would be willing to accept the security risk of their competitors' getting hold of their transaction information? Most companies are very secretive about this sort of thing – how might it be in their interest to share it around?

If, on the other hand, databases were only kept by single retailers, how would this differ from loyalty card schemes, which are massive databases of buying habits with our complete agreement?

The main reason retailers are interested in RFID is because it gives them inventory information in better ways than older technologies did. Once the item is sold, they couldn't care less.

To be sure, some have introduced loyalty card type schemes based on RFID cards which store customer information. But this is fully in the open, with the customer's acceptance, because they get advantages, such as targeted specials. But if the customer doesn't want it, he has the choice of not accepting it.

The point to keep in mind when reading doomsday scenarios about RFID usage is who benefits from information gathering and who has to be involved for it to happen. We don't believe that use of RFID for customer tracking is different from any number of already existing systems, such as loyalty cards or credit cards which are hugely effective at gathering information about customers. And these existing systems do not require massive investment in infrastructure and the complicity of uninterested middle parties, like small retailers.

Big Brother already knows what you are doing. If you don't want him to know, you can take steps to limit the information that is being recorded about you. You would take the same steps in an RFID information tracking environment.

This technology does not make possible any significant increase in invasion of privacy beyond what already exists.

There are 3 main aspects of our use of Python within our software development:

1. The API is socket-based and uses Python commands entirely
2. The embedded reader application is Python-based
3. The GUI application development is done with PyQt

We decided early on to make our application interface be via a network socket and to use plain text for all data formatting. This idea was inspired by the way many Internet services, such as DNS, mail and the Web work. It removes issues of programming language, operating system or CPU-dependence and makes the system easy enough to use with a telnet client.

It also allows readers and client applications to be connected in any manner the system integrator wants to. We aim to make the readers easy to use and as flexible as we can possibly make them.

Many of our readers are embedded Linux-based, giving us great flexibility in networking and ancillary services, such as use of FTP for application upgrades. Some smaller readers run the server application on Windows or Linux on desktop PCs to provide the socket server. Much of the application code is common between the two types.

By keeping the same interface, a socket, and the same API, we make developers' lives easier since they only have to develop applications once, which will run on all readers, even future models that don't exist yet.

In our earlier reader application, which was C++-based, we discovered there were a number of disadvantages:

• The amount of code needed to support the binary-formatted packets, along with the communications protocol and support, was more than half the entire programming.
• Every time we wanted to add a new function, it meant a considerable amount of effort to add the API hooks for it.
• Whenever a tag was sighted, there was a communications lag to the client system both ways, in addition to the processing overhead in the client system before the next tag command could be sent. In many cases, users were running Windows as the client system, which meant that real-time performance was not acceptable.
• Use of binary formatting of data packets creates issues of byte alignment of different CPUs and structure packing in different programming languages and systems.

After some consideration, it was decided that a way to resolve all these problems was to use a scripting language as the API and do all transactions in plain text. Some investigation of different scripting languages was done.

At that time (3 years ago), Embedded Java did not exist, and the standard Java environment was too large and seemed too difficult to hook into compiled C/C++ modules.

Perl had a wide following, but did not scale well or support object-oriented programming well. Also, it did not meet the software department's sense of style and elegance.

Tcl was too limited and other language options simply didn't have the following or active development to inspire confidence.

All of which led us to Python. As long-time programmers in languages such as C++ and Borland Pascal, we found a kindred spirit in Python, a language of simplicity and elegance with considerable power.

The API is essentially a large set of function calls to access the functionality of the reader. A programmer has direct access to control the hardware. For example, sending this command over a socket connection will turn the transmitter off (useful if you're testing or doing maintenance):

Power(0)

There is one key component of the software design. We get end user programmers to send a series of tag commands and small Python scripts to the reader to define what actions to take whenever a new tag is sighted.

Because these scripts have access to the full functionality of the reader, it allows programmers to customise the reader to do exactly what they want. Because the scripts are run on the reader, rather than remotely, it means the readers have the highest possible performance.

Here is a simple example of defining a series of tag commands, to be executed each time a new tag is sighted:

Actions(Command(readAddress=12,readLength=8),
        Command(readAddress=14,readLength=2,
               writeAddress=14,writeData=[0x1200,0x1400]))

In this example, a block 8 words long is read from address 12, then data is written to address 14. This list can be extended at will . Formatting is just for ease of reading here.

Note how named parameters are used. This gives users great flexibility in creating tag commands, making it easier to write the command and to remember the syntax.

User-defined scripts are, of course, able to define their own variables and functions, which allows complex behaviour to be defined.

An example of a simple script would be to read a block of memory from the tag, then compare some part of the result with a known value, then make a decision about what to send to the tag on that basis. This sort of thing is very easy to do, as follows:

postscript="""if state.Reply().Data()[0] != 0xffff:
     Message('Word 10 is not empty')
else:
     Message('Word 10 is empty')\n"""

InterrogateAction(
     (None, Command(readAddress=10,readLength=1), postscript))

The InterrogateAction defines what command will always be sent when the reader is looking for new tags. It passes a tuple consisting of (script to run before sending the command each time, the command to send, script to run after waiting for a reply).

In this example, the post interrogate script is defined as a string variable, to make the example more readable. Here the data read back from the tag is looked at and a decision is made as to what message to display as a result.

Some of the reader scripts we have written have had over a page of text, which doesn't make for easy readability, so we have a GUI application to aid in the development of these scripts.

We originally intended to develop the core of the application in C++ and just use Python for the client API-related sections. But terrifying deadlines and limited resources forced us to consider the use of Python for the main application.

From the outset, it was intended that this would be a temporary step to allow for rapid development. For 3 years, already, however, 80% of the reader application code has remained written in Python.. This has proven invaluable in giving us a rapid development platform to try different ideas to get the best performance and functionality. We are now going through the work of converting the existing Python “engine” of the application into C++ to get the best performance.

We are using Boost.Python to provide the interface between Python and C++. Though it has a performance and size cost, the transparent use of C++ classes and removal of Python C API tasks such as reference counting makes development a great deal easier.

Using Python has proven invaluable for application development. The ease of making structural changes has meant that the application has been able to evolve rapidly to a mature state. Nonetheless, in the end, it simply hasn't proven fast enough for us since our requirements are extremely high: we want to be able to process over a thousand tags a second.

One of the key problems facing the reader software was being able to operate in a number of different states, where key actions took place in vastly different sequences. This is driven by the major mode and the minor post-processing priority. The software was set up to decode and act on tag replies as a background task, although the priority of this task could vary depending on the application.

The solution was to use a table of references to functions, as follows:

# shortcuts for class functions for each major action
I = self._InterrogateState
S = self._SpecificState
M = self._MuteState
B = self._BreakState
P = self._PostProcessState
A = self._AxisState
# list of function priorities for all combinations of mode and post process
modePriority = [
     # Mode 0 – phased interrogation phase, then specific command phase
     [ M, I, S, A ], # low priority post process
     [ M, I, S, P, A ], # medium priority post process
     [ P, M, I, S, A ], # high priority post process
     # Mode 1 - interleaved interrogation and specific command
     [ S, M, I, A ], # low priority post process
     [ S, M, I, P, A ], # medium priority post process
     [ P, S, M, I, A ], # high priority post process
     # Mode 2 – Stack tags
     [ S, M, I, B, A ], # low priority post process
     [ S, M, I, P, B, A ], # medium priority post process
     [ P, S, M, I, B, A ], # high priority post process
]

Which line of this table is to be used is decided based on the settings of 2 variables, as follows:

self.mainLoop = modePriority[self.sequenceMode * 3 + self.sequencePriority]

Then the main loop of the application becomes simply this:

while not self.application.terminate:
     # iterate through the current mode priority list calling functions
     # until one of them returns non-zero
     for item in self.mainLoop:
         if item():
             break # non-zero returned, so exit loop

In this way, we effectively have a priority queue of actions, which is dynamically driven by the data in various queues used by each of these actions. It is very simple, fast, and flexible.

We develop many GUI applications, for demonstrations, client applications, engineering tests and so on. All of this development is done in PyQt – that is, Python over the Qt library. We currently have about 750K of Python code in this application.

Even though Qt is commonly associated with KDE, it is the best cross-platform GUI toolkit we have ever encountered. Many other GUI toolkits act as translation layers to the underlying native API, which results in their having only the common functionality across all operating systems, which is not good.

Qt grabs an area of raw screen from the underlying OS and then implements everything itself, which means there are no subtle translation difficulties. Functionality such as resizeable windows is trivially easy in Qt, but painful in most other toolkits. Everything we have wanted to do so far has been straightforward to implement.

The PyQt implementation is extremely good. Because Qt is developed in C++, it translates to Python classes extremely transparently.

We use BlackAdder (from TheKompany) as an IDE. It gives us royalty-free distribution to Linux and Windows.

We like the idea that we get cross-platform development without any extra work being required of us. For those who don't like working in Windows, all development can be done in Linux, or vice versa. PyQt has been ported to Mac OS X, so we hope that this will become available through BlackAdder eventually.

Using the Qt designer, we can graphically design our dialogs, which are saved as XML files. These are compiled by PyQt into Python files. We then edit our source files to inherit from the classes defined in these compiled Python files and implement the functionality we want.

Our application consists of a lot of windows which each perform a different demonstration or test. So there is a need for a common communications system to handle commands and replies, doing all the common work.

We use the Qt class QSocket as a starting point, using regular expressions to decode messages we are interested in, such as tag replies. The replies include a subset of a number of different fields, based on what kind of reply it is. Here is an example:

Info,Reply:Timestamp 24ff, SpecificID 7ef05d1d, ReadAddress 000a, Data: 1000 2000 3000

The following is a portion of the reply handling code we use, to show how we use the power of regular expressions:

class MonitorSocket(QSocket):
   def __init__(self, parent=None, name= "MonitorSocket"):
      QSocket.__init__(self, parent, name)
      self.connect(self, SIGNAL("readyRead()"), self.ReadyRead)
      # Detect log message format
      self.messageRe = re.compile(
      r"(?:(?P<time>\w\w\w \d\d \d\d:\d\d:\d\d\.\d\d\d):)?(?P<severity>\w+),
      (?P<module>\w*):(?P<message>.*)")
      # Define some convenient re expressions to decode tag replies
      # regular expression to find and decode data fields from tag reply and
      # expired messages
      self.replyRe = re.compile(
      r'Info,(?:Tag:Expired (?:no response )?(?P<expired>[\dA-F]{8}))|'+\
      r'(?:Reply:'+\
      r'(?:Axis (?P<axis>\w), '+\
      r'Channel (?P<channel>\w), )?'+\
      r'Timestamp (?P<timestamp>[\dA-F]{4}), '+\
      r'(?:Hardcode (?P<hardcode>[\dA-F ]*), )?'+\
      r'(?:MemorySize (?P<memorySize>[\dA-F]{4}), )?'+\
      r'(?:BlockSize (?P<blockSize>[\dA-F]{4}), )?'+\
      r'(?:SubBlockSize (?P<subBlockSize>[\dA-F]{4}), )?'+\
      r'(?:EraseWriteTime (?P<eraseWriteTime>[\dA-F]{4}), )?'+\
      r'(?:LockPointer (?P<lockPointer>[\dA-F]{4}), '+\
      r'Manufacturing (?P<manufacturingNumber>[\dA-F]{4}), )?'+\
      r'SpecificID (?P<id>[\dA-F]{8})'+\
      r'(?:, GroupID (?P<groupID>[\dA-F]{4})'+\
      r', ConditionalID (?P<conditionalID>[\dA-F]{4})'+\
      r', Configuration (?P<configuration>[\dA-F]{4}))?'+\
      r'(?:, CRC (?P<crc>[\dA-F]{8}))?'+\
      r'(?:, ReadAddress (?P<address>[\dA-F]{4}), '+\
      r'Data: (?P<data>[\dA-F ]*))?)', re.I)

# Data has been received. Process a line at a time and send signal to app
   def ReadyRead(self):
      while self.canReadLine():
         line = str(self.readLine())
         match = self.messageRe.search(line) # is it a message line?
         if match: # this line is a message
            dict = match.groupdict() # retrieve all the decoded fields
            dict.update({ 'line' : line }) # append the whole 'line'
            match = self.replyRe.search(line) # is it a reply
            if (match):
               dict.update(match.groupdict()) # append these tag fields
               self.emit(PYSIGNAL("receivedReply()"), (dict,))
            else:
               # any other log
               self.emit(PYSIGNAL("receivedMessage()"), (dict,))
               # unformatted responses
               self.emit(PYSIGNAL("receivedOther()"), (line,))

What this code does is to strip a reply into its pieces and then send the dictionary of those pieces as a signal, which is received and processed by all windows interested in that particular kind of reply. This design makes it very easy to write new windows because it makes handling replies trivially easy. The following example shows how the reply data can be received and processed.

This demonstration was for a prototype gambling chip reader we built. The image shows the appearance of the actual reader. The 2 yellow circles mark the position of reader antennas beneath the surface. The aim of the demonstration was to show the count of tags on each circle.

The first phase was to design the window layout using Qt Designer, then generate Python code from it, as follows:

class GamblingForm(QWidget):
   def __init__(self,parent = None,name = None,fl = 0):
      QWidget.__init__(self,parent,name,fl)
      if not name:
         self.setName("GamblingForm")
      self.setSizePolicy(QSizePolicy(5,0,0,0,self.sizePolicy().hasHeightForWidth()))
      self.setMinimumSize(QSize(560,600))
      self.setMaximumSize(QSize(560,600))
      self.xAxis = QLabel(self,"xAxis")
      self.xAxis.setGeometry(QRect(360,390,110,130))
      self.xAxis.setPaletteBackgroundColor(QColor(0,235,0))
      xAxis_font = QFont(self.xAxis.font())
      xAxis_font.setPointSize(48)
      self.xAxis.setFont(xAxis_font)
      self.xAxis.setTextFormat(QLabel.PlainText)
      self.xAxis.setScaledContents(1)
      self.xAxis.setAlignment(QLabel.AlignCenter)
      self.yAxis = QLabel(self,"yAxis")
      self.yAxis.setGeometry(QRect(80,390,110,130))
      self.yAxis.setPaletteBackgroundColor(QColor(0,235,0))
      yAxis_font = QFont(self.yAxis.font())
      yAxis_font.setPointSize(48)
      self.yAxis.setFont(yAxis_font)
      self.yAxis.setTextFormat(QLabel.PlainText)
      self.yAxis.setScaledContents(1)
      self.yAxis.setAlignment(QLabel.AlignCenter)
      self.languageChange()
      self.resize(QSize(560,600).expandedTo(self.minimumSizeHint()))
      self.clearWState(Qt.WState_Polished)

   def languageChange(self):
      self.setCaption(self.__tr("Gambling Reader Demonstration"))
      self.xAxis.setText(self.__tr("000"))
      self.yAxis.setText(self.__tr("000"))

   def __tr(self,s,c = None):
      return qApp.translate("GamblingForm",s,c)


The next stage was to edit the implementation of this design. This is a slightly simplified version of the actual file:

from qt import *
from GamblingForm import GamblingForm
from ToolBase import ToolBase # define a common support class

class GamblingDialog(GamblingForm, ToolBase):
     # define the commands to be sent to the reader to configure it
     initCmd='AxisBreak(1)\n\rFullReplyFormat(1)\n\rAxisSwitching(1)\n\r
ToggleReaderIDEvery(700)\n\rTagTimeout(5000)\n\r'
   def __init__(self, parent=None, name='GamblingDialog'):
     ToolBase.__init__(self, parent, name, GamblingForm, init)
     self.setPaletteBackgroundPixmap(QPixmap('%sBackground.png' % baseDir))
     self.setIcon(QPixmap('%sGambling.png' % baseDir))
     self.tagCount = [0, 0] # number of active tags on each axis
     self.knownTags = {} # dictionary of current tags indexed by ID
     self.xAxis.setText('0')
     self.yAxis.setText('0')
     # connect to tag replies being sent as a signal
     self.MonitorConnect("receivedReply()", self.Received)

   def Received(self, message): # Process a single tag reply
     id = message.get('expired') # lookup this field of the reply
     if id: # If the tag has expired, then forget it & reduce our count
        try:
         axis = self.knownTags[id]
         del self.knownTags[id]
         self.tagCount[axis] -= 1
        except KeyError: # Unknown tag has expired
         pass
     else:
        id = message['id']
        # Left circle appears as 'X' and right circle as 'Y' axis
        axis = ord(message.get('axis')) - ord('X') # axis to number
        self.tagCount[axis] += 1 # increment number of tags on this axis
        if self.knownTags.has_key(id): # tag has moved between axes
           oldAxis = self.knownTags[id]
           if oldAxis != axis:
              self.tagCount[oldAxis] -= 1
        self.knownTags[id] = axis
     # Update the score on the screen
     self.xAxis.setText(str(self.tagCount[0]))
     self.yAxis.setText(str(self.tagCount[1]))


This demonstration takes very little code to implement. The ToolBase class provides a common infrastructure for all tools to use, making the work even less for each tool. Extracting tag reply information and updating the screen is remarkably easy.

The ease of this process means that a marketing request for a new demonstration or engineering test tool can be quickly met. The power and simplicity of Qt along with the infrastructure of the application means we have a great development environment.

 Py is committed to bringing you great Python Articles.