Py - The Online Python Zine

				Issue 5 - Revision 7 / April 20, 2004


	Py Links: Latest Issue Issue 08 Issue 07 Issue 06 Issue 05 Issue 04 Issue 02 Issue 01





	Articles: Throughout the quarter we cover topics of interest to Python developers. RSOAP Simple Code Generation 4ss Pyro PyCon 2004 Kaa & Firedrop XML-RPC for Python Applied XML-RPC

Applied XML-RPC
- Painless Web Publishing
- - - - - - - - - - - -
By Phillip Pearson | March 16, 2004

Introduction

XML-RPC is a simple, cross-platform way to call functions on remote machines. Developed by UserLand Software in 1998, XML-RPC provides way to interface with applications running on UserLand's Frontier server. XML-RPC is a predecessor of SOAP, a similar protocol currently being standardized by the W3C and supported enthusiastically by Microsoft, SUN, IBM, and the big players in the web services space.

Are people actually using XML-RPC, and, if so, what are they doing with it? Not surprisingly, one of the most widely deployed XML-RPC applications is from UserLand: a weblogging tool called Radio, which is designed to be much kinder on the server than most blogging tools. Instead of generating the UI on the server, a client runs on the user's PC, and all the server needs to do is receive HTML files and serve them back out.

A specification for an early version of xmlStorageSystem, the XML-RPC interface Radio uses to publish files to the server, is publicly available and is quite easy to use from Python. Based on this spec, I have written a server (Python Community Server) that is compatible with Radio and UserLand's Radio Community Server package. In this article I will summarize the features offered by ``community servers'' --- servers implementing the xmlStorageSystem interface --- and demonstrate their use.

Py readers are welcome to use my server, at http://www.pycs.net/, for testing and (within reason) hosting of web pages you produce. Make sure you change the email addresses, names and passwords given here before running your code.

Getting Started

First, import the module xmlrpclib and prepare a server proxy object:


>>> import xmlrpclib

>>> s = xmlrpclib.Server('http://www.pycs.net/RPC2')

Most functions require a login ID. The function xmlStorageSystem.registerUser() will create a user account on the server, keyed by your email address, and return a usernum, which, along with the password you supply, will identify you to the server for all future calls.


>>> email = 'pp@myelin.co.nz'

>>> name = 'Phillip Pearson'

>>> import md5

>>> password = md5.md5('password').hexdigest()

>>> r = s.xmlStorageSystem.registerUser(email, name, password, 80, 'Python')

>>> r

'message': 'Good to have you here, Phillip Pearson!', 

'flError': <Boolean False at 8ecf48>,

'usernum': '0000048'

The 'False' value of r['flError'] indicates that the call has succeeded and that some web space is now available for our use. A sensible thing to do at this point is to save the usernum for future use.


>>> usernum = r['usernum']

>>> usernum

'0000048'

>>> open('usernum.txt', 'w').write(usernum)

From now on, we can easily recall it:


>>> usernum = open('usernum.txt').readline()

Sending Files

Now that we have created an account, we can send files to the server. Let's post an index, so someone visiting our new site will see something useful. First, we make a basic index page.


>>> indexHtml = '''<html>

...  <head>

...   <title>Hello!</title>

...  </head>

...  <body>

...   <h1>Placeholder</h1>

...   <p>There might be a weblog here, one day ... for the moment, 

hello from %s!</p>

...  </body>

... </html>''' % name

We use the function xmlStorageSystem.saveMultipleFiles() to store the file remotely; it saves one or more files in the public web space allocated to us when we registered with the server. This call takes the usernum we received from xmlStorageSystem.registerUser(), our password, a list of filenames, and a list of file contents. In our case, we only have one file to send, so the filename and file content lists will be single-element lists containing, respectively, ``index.html'' and the HTML sourcecode of our index page.


>>> r = s.xmlStorageSystem.saveMultipleFiles(usernum, password, ['index.html'], [indexHtml])

>>> r 

'yourUpstreamFolderUrl': 'http://www.pycs.net/users/0000048/',
 
'urlList': ['http://www.pycs.net/users/0000048/index.html'],

'message': '1 files have been saved for you, Phillip Pearson',

 'flError': <Boolean False at 8ecf48>

The return code from the function xmlStorageSystem.saveMultipleFiles() reveals some useful information about the file we just saved. r['yourUpstreamFolderUrl'] is the top-level URL of our webspace on the community server; all the paths we pass to saveMultipleFiles() are relative to it. r['urlList'] contains a list of the URLs of all the files that have been saved on the server. In this case, there is only one.

If an error had occurred during the save operation, a blank URL would have appeared in r['urlList'] and r['message'] would have contained an error message. We can see this if we try to save a file above the root:


>>> r = s.xmlStorageSystem.saveMultipleFiles(usernum, password, ['../index.html'], [indexHtml])

>>> r 

'yourUpstreamFolderUrl': 'http://www.pycs.net/users/0000048/', 

'urlList': [''],

 'message': "Security warning: '..' found in filename", 

'flError': <Boolean True at 819f36c>

Removing Files

Another useful function is xmlStorageSystem.deleteMultipleFiles(), which allows the user to delete files from the public webspace. It takes as arguments the usernum, password, and a list of files to delete. The following creates and then deletes a dummy file:


>>> s.xmlStorageSystem.saveMultipleFiles(usernum, password, ['test.xml'], ['<:?xml version="1.0"?><hello/>'])

{'yourUpstreamFolderUrl': 'http://www.pycs.net/users/0000048/', 

'urlList': ['http://www.pycs.net/users/0000048/test.xml'], 

'message': '1 files have been saved for you, Phillip Pearson',

 'flError': <Boolean False at 8ecf48>}

 >>> s.xmlStorageSystem.deleteMultipleFiles(usernum, password, ['test.xml'])

{'message': '1 files have been deleted for you, Phillip Pearson', 

'flError': <Boolean False at 8ecf48>,

 'errorList': ['']}

* Next steps

Now that we can store files on the community server and delete them, we can take advantage of the extra information community server provides. Basically, this is everything Radio needs to be able to render the HTML for a weblog.

Before we get going on that, let's create a function, pushIndex(), that will allow us to quickly regenerate the index and push it up to the server, to avoid unnecessary typing later on.


 >>> indexHead = '''<html>

...  <head>

...   <title>Hello!</title>

...  </head>

...  <body>

... '''

 >>>

 >>> indexBody = '''<h1>Placeholder</h1>

...   <p>There might be a weblog here, one day ... for the moment, hello from ''' + name + '''!</p>

... '''

 >>>

 >>> indexFoot = ''' </body>

... </html>'''

 >>>

 >>> def pushIndex():

...     r = s.xmlStorageSystem.saveMultipleFiles(usernum, password, ['index.html'], [indexHead + indexBody + indexFoot])

...     if r['flError']:

...             raise "Error saving files: %s" % ( r['message'], )

...     return r['urlList'][0]

and test it:


 >>> pushIndex()

'http://www.pycs.net/users/0000048/index.html'

Good - that worked. From now on, calling pushIndex() will rebuild index.html and push it up to the server. As we add more HTML to indexBody, we will be able to see our changes by leaving a browser window open at http://www.pycs.net/users/0000048/ and hitting the refresh button after each call to pushIndex().

Hit tracking

The community server provides, among other things, a rudimentary hit tracking system. To use it, each page must load a counter image.

The xmlStorageSystem.getServerCapabilities function returns a host of information, including the URL of the script that generates the counter image, which is returned in the 'webBugUrl' field.


 >>> cap = s.xmlStorageSystem.getServerCapabilities(usernum, password)

 >>> cap['webBugUrl']

'http://www.pycs.net/system/count.py'

The script takes as its arguments our usernum, a weblog group (normally 'default'), and a referrer URL. We generate a call to the script with the following snippet of JavaScript.


 >>> bugScript = '''<script language="javascript"><!--

...     document.write('<img src="''' + cap['webBugUrl'] + 
'''?group=default&usernum=''' + usernum + '''&referer=' + 

escape(document.referrer) + '" height="1" width="1">');

...   //--></script>

... '''

Add and upload as usual:


 >>> indexFoot = bugScript + indexFoot

 >>> pushIndex()

'http://www.pycs.net/users/0000048/index.html'

Now our placeholder page will show up in the site rankings and have its own referrer tracking page. The URLs of these pages are returned from xmlStorageSystem.getServerCapabilities alongside the counter image URL:


>>> cap['urlRankingsByPageReads']

'http://www.pycs.net/system/rankings.py'

 >>> cap['urlReferers']

'http://www.pycs.net/system/referers.py?usernum='

Visiting the rankings page, we see our placeholder page, but it has no name! To fix this, we need to call the xmlStorageSystem.ping function and tell the community server a little more about ourselves.


 >>> blogTitle = 'Py article examples for ' + name

 >>> ping = s.xmlStorageSystem.ping(usernum, password, 0, 80, {'email': email, 'weblogTitle': blogTitle, 'serialNumber': 'none',
'organization': 'poor', 'flBehindFirewall': xmlrpclib.True, 'name': name})

Revisiting the rankings page, our placeholder page should now be visible under the name "Py article examples for (your name here)".

Weblog updates

We have one more feature to implement before our placeholder page can be considered a true 'community member': weblog update tracking. As with the hit counting, the URL is returned from xmlStorageSystem.getServerCapabilities:


>>> cap['urlWeblogUpdates']

'http://www.pycs.net/system/weblogUpdates.py'

To show up on this page, we need to 'ping' the community server to tell it that we have finished updating. To do this, we call a function from the Weblogs.Com XML-RPC API[7], weblogUpdates.ping. This takes as arguments the weblog title and the URL of the main page.


 >>> s.weblogUpdates.ping(blogTitle, cap['yourUpstreamFolderUrl'])
'message': 'Thanks for the ping!', 
'flError': <Boolean False at 819e8b4>

Checking the community server updates page (the URL we saw before), our placeholder should show up as "Py article examples for (your name here)".

To fit in with the rest of the world, we now call the same method on Weblogs.Com:


>>> xmlrpclib.Server( 'http://rpc.weblogs.com/RPC2' ).weblogUpdates.ping(blogTitle, cap['yourUpstreamFolderUrl'])
'message': 'Thanks for the ping. We checked and found that the "Py article examples" weblog has changed, so it will appear in <a href="http://www.weblogs.com/changes.xml">changes.xml</a> next time it is updated.',
'flerror': <Boolean False at 819e8b4>

Our placeholder will soon appear right at the top of the front page of Weblogs.Com.

All of the above

One exceedingly useful service the community server provides is comment hosting. The radioCommunityServer.getInitialResources function returns the URL to a script that we can use to tag a 'comment' link wherever we like on our site.


 >>> indexFoot = '<p><a href="%s">updates</a> | <a href="%s">rankings</a> | <a href="%s%s">referrers</a>\n' % ( cap['urlWeblogUpdates'], cap['urlRankingsByPageReads'], cap['urlReferers'], usernum ) + indexFoot
 >>> pushIndex()
'http://www.pycs.net/users/0000048/index.html'

We now have all the necessities for a weblog that is part of a community. But wait - that's not all! There are more features that make publishing to a community server over XML-RPC a much more attractive option than sending files over FTP to a standard web server.

Comments


 >>> res = s.radioCommunityServer.getInitialResources()

 >>> res['commentsPageUrl']

'http://www.pycs.net/system/comments.py'

The comment script takes two arguments: 'u', our usernum, and 'p', the ID of the comment thread. Let's add a comment link to the bottom of the placeholder page.


>>> commentLink = '<p><a href="%s?u=%s&p=%d">Comment on this page</a></p>' % (res['commentsPageUrl'], usernum, 1)

We then add this to our HTML, as usual, and upload the new file:


 >>> indexBody += commentLink + "\n"

 >>> pushIndex()

'http://www.pycs.net/users/0000048/index.html'

Reloading the page now will reveal the new comment link, which should lead to an empty comments page.

To make comments more interesting, we can display the number of comments on the front page. To do this, we include a bit of JavaScript that is generated by the comments page when we call it with an argument of 'c=counts'. This defines a function, commentCounter, that we can call from inside our comment link to display the comment count for a given comment thread ID.


 >>> indexHead += '<script language="javascript" src="%s?u=%s&c=counts"></script&gr;\n' % (res['commentsPageUrl'], usernum)

 >>> indexBody += '<p>(<script language="javascript">commentCounter(1)</script> so far)</p>'

 >>> pushIndex()

'http://www.pycs.net/users/0000048/index.html'

The page will now inform all visitors that there are zero comments. Hooray!

Spam-reducing mail relay

One more convenience is the spam-reducing mail relay, less accurately known as the "spam-free mailto page". As the community server knows our e-mail address, it can act as a proxy, passing on messages entered into an HTML form on the server. This will provide fairly reasonable protection against spam until somebody finds it worthwhile to write a script to specifically attack weblog mail relay pages.

Once again, the URL is in the data returned from:


xmlStorageSystem.getServerCapabilities:



 >>> cap['urlSpamFreeMailto']

'http://www.pycs.net/system/mailto.py'

Adding a link to this to our HTML and uploading as usual makes our placeholder page ever more sophisticated.


>>> indexBody += '<p><a href="%s?usernum=%s">Send an e-mail to the author</a></p>\n' % ( cap['urlSpamFreeMailto'], usernum )

 >>> pushIndex()

'http://www.pycs.net/users/0000048/index.html'

Conclusion

This article has briefly introduced most of the features of weblog community servers such as UserLand's Radio Community Server[8] and the author's Open Source Python Community Server[9].

The example code given in this article, along with Mike Soulier's simple template processing method from Py 01.01, could be taken as the starting point for writing a weblogging application along the lines of the author's own bzero[10].

Alternatively, how about creating something new and interesting that uses the xmlStorageSystem API? One possibility would be to follow the original design plan for xmlStorageSystem, which was envisaged as a competitor to Microsoft Passport.

Another option is to experiment directly with Radio UserLand: it can be extended with Python, and its built-in outline editor can be used as a Python IDE.

For further Reference:

XML-RPC http://www.xmlrpc.com/

xmlStorageSystem spec http://www.soapware.org/xmlStorageSystem

Weblogs.Com RPC spec http://www.xmlrpc.com/weblogsCom

Radio Community Serverhttp://rcs.userland.com/

Python Community Server http://www.pycs.net/

bzerohttp://www.myelin.co.nz/bzero//a>

Phillip Pearson

Phillip Pearson (pp@myelin.co.nz) is a software developer (Python/C/C++/Java/Perl/C#) and Python evangelist from New Zealand. Using Python, he has developed Python Community Server, an Open Source weblog server, and bzero, a weblogging application. He uses Python to download and analyse over 22,000 weblogs for the Blogging Ecosystem, an online link analysis exercise that seems to have turned into a huge popularity contest. He also runs the Internet Topic Exchange, a shared categorisation system for weblog posts. He has a weblog, "Second p0st", at http://www.myelin.co.nz/post/.

Py is committed to bringing you great Python Articles.

Home Subscribe Migration FAQ Contact PyZine Write for PyZine ZopeMag opensourcexperts.com