EuroPython 2014 pyRserve-slides now available online

September 19, 2014 at 11:50 AM | categories: Python, R, pyRserve | View Comments

After our well-attended Python Meetup last Wednesday in Heidelberg about "Connecting Python to other programming languages for scientific computing" I had been approached to publish the slides of my pyRserve talk (which has been the same as the one I've given at EuroPython in Berlin, for those who are interested). These are now online, so feel free to download them.

Read and Post Comments

pyRserve 0.8.1 released

July 18, 2014 at 10:20 AM | categories: Python, R, pyRserve | View Comments

Just in time for EuroPython in Berlin a new version of pyRserve just got released yesterday. If you happen to come to the conference there will be an intoductory talk on pyRserve on Fri 25h of July titled "Combining the powerful worlds of Python and R". Would be great to see you there.

About pyRserve

pyRserve is a (pure python) client for connecting Python to an R process on a remote server via TCP-IP (using Rserve). R is one of the most important and most widely spread open source statistic packages available.

Through such a pyRserve connection the full power of R is available on the Python side without programming in R. From Python variables can get set in and retrieved from R, and R-functions can be created and called remotely. No R-related libraries need to be installed on the client side, pip install pyRserve is all that needs to be done.

Sample code

This code assumes that Rserve (the part that connects the R engine to the network) already is running. Details can be found in the pyRserve docs.

>>> import pyRserve
>>> conn = pyRserve.connect('Rserve.domain.com')
>>> conn.eval('sum( c(1.5, 4) )') # direct evaluation of a statement in R
5.5
>>> conn.r.myList = [1, 2, 3] # bind a Python list in R to variable 'myList'

>>> conn.voidEval('func1 <- function(v) { v*2 }')  # create a function in R
>>> conn.r.func1(4)                                # call the function in R
16

Most important changes in V 0.8.x

  • Support for out-of-bound messages (allows for callbacks from R to Python) (contrib. by Philip. A.)
  • Rserve can now be shutdown remotely (contrib. by Uwe Schmitt)
  • Fixed bug when passing R functions as parameters to R functions
  • Documentation errors have been fixed

Documentation and Support

The documentation for pyRserve is available at http://packages.python.org/pyRserve

The corresponding Google news group can be found at http://groups.google.com/group/pyrserve

Read and Post Comments

Publish-Subscribe with web sockets in Python and Firefox

July 22, 2012 at 04:20 PM | categories: Python, Web | View Comments

WebSockets provide a way to communicate through a bi-directional channel on a single TCP connection. This technology is especially interesting since it allows a web server to push data to a browser (client) without having the client to constantly poll for it. In contrast to normal HTTP requests where a new TCP connection gets opened and closed for each request web socket connections are kept open until one party closes them. This allows for communication in both directions, and calls can be made multiple times on the same connection.

In this little article I basically combine what I found on Sylvain Hellegouarch's documentation for ws4py (a WebSocket client and server library for Python) and the article HTML5 Web Socket in Essence by Wayne Ye.

More specifically the examples below shows how multiple clients subscribe via websockets to a cherrypy server through a web socket connection. The first of the two clients in the example below is a very lightweight client based solely on the ws4py package, the other (javascript) implementation is supposed to run in Firefox.

The server

This example provides a minimal publishing engine implemented with cherrypy. An instance of class WebSocketTool is hooked up into cherrypy as a so-called cherrypy tool, and a web socket handler (the Publisher-class) is bound to this tool as a handler for calls to the path /ws:

import cherrypy
from ws4py.server.cherrypyserver import WebSocketPlugin, WebSocketTool
from ws4py.websocket import WebSocket

cherrypy.config.update({'server.socket_port': 9000})
WebSocketPlugin(cherrypy.engine).subscribe()
cherrypy.tools.websocket = WebSocketTool()

SUBSCRIBERS = set()

class Publisher(WebSocket):
    def __init__(self, *args, **kw):
        WebSocket.__init__(self, *args, **kw)
        SUBSCRIBERS.add(self)

    def closed(self, code, reason=None):
        SUBSCRIBERS.remove(self)

class Root(object):
    @cherrypy.expose
    def index(self):
        return open('ws_browser.html').read()

    @cherrypy.expose
    def ws(self):
        "Method must exist to serve as a exposed hook for the websocket"

    @cherrypy.expose
    def notify(self, msg):
        for conn in SUBSCRIBERS:
            conn.send(msg)

cherrypy.quickstart(Root(), '/', 
    config={'/ws': {'tools.websocket.on': True,
                    'tools.websocket.handler_cls': Publisher}})

The only purpose of the Root.ws()-method is to make this method available under /ws in the web server through the cherrypy.expose decorator. Whenever a websocket client makes a request to /ws an instance of class Publisher is created, which registers itself to the global SUBSCRIBERS set on __init__(). When the server goes down, or the client disconnects, its closed() method is called.

The only packages needed for this example are cherrypy and ws4py. Both can be easily installed via easy_install or pip. Save the code above as ws_server.py and start it with

python ws_server.py

Now the server is ready to accept client connections through the web socket protocol. As soon as one of the clients described below has subscribed to this server messages can be published by calling the Root.notify() method. Since it is exposed it is possible to call it from the command line with

curl localhost:9000/notify?msg=HelloWorld

Of course wget works as well.

A pure Python client

The Python client's code is quite short. ws4py provides three sample client implementations, the threaded one has been chosen for this example. The others are using gevent or Tornado.

from ws4py.client.threadedclient import WebSocketClient

class Subscriber(WebSocketClient):
    def handshake_ok(self):
        self._th.start()
        self._th.join()

    def received_message(self, m):
        print "=> %d %s" % (len(m), str(m))

if __name__ == '__main__':
    ws = Subscriber('ws://localhost:9000/ws')
    ws.connect()

The method handshake_ok() has been overridden to keep the thread stored in self._th running continuously (the original implementation quits after one second). After the Subscriber-class has been instantiated it connects to the cherrypy server. Whenever the server sends a message it will be delegated to the method received_message() where it gets printed to stdout.

Just store this code into a file, e.g. ws_subscriber.py and start it in from a new shell. The cherrypy server should print a message to the console that it received a web socket connection.

Now again call the notify-method in the server:

curl localhost:9000/notify?msg=HelloWorld

and the python client should print your message to the screen.

A web socket client in Firefox

This browser client uses the web socket protocol built into Firefox. The example below works for me in FF14, it failed for FF8. I'm not sure which version of Firefox starts to support it. Safari version 5.0 also fails. IE has not been tested.

<html>
  <head>
    <script>
      var websocket = new WebSocket('ws://localhost:9000/ws');
      websocket.onopen    = function (evt) { console.log("Connected to WebSocket server."); };
      websocket.onclose   = function (evt) { console.log("Disconnected"); };
      websocket.onmessage = function (evt) { document.getElementById('msg').innerHTML = evt.data; };
      websocket.onerror   = function (evt) { console.log('Error occured: ' + evt.data); };
    </script>
  </head>
  <body>
    <h1>Websocket demo</h1>
    Message: <span id="msg" />
  </body>
</html>

At load time a Websocket-instance is created and event handlers are installed. The interesting one is the onMessage-handler: it is called for each message received, it copies the message into the SPAN element and thus makes it visible.

Make sure to store this html page in the same directory as the ws_server.py above since we are going to open it through cherrypy's index method. For this to work it has to be named ws_browser.html. Now open Firefox and direct it to http://localhost:9000. You should immediately see this page. The SPAN element should be empty.

Again repeat the curl or wget command in your shell and both the python client (if it is still running) and the SPAN element should display your "HelloWorld" message.

Read and Post Comments

pyRserve 0.6.0 released

July 04, 2012 at 07:20 PM | categories: Python, R | View Comments

While being at EuroPython in Florence the latest version of pyRserve has now been finished and is available via pypi (easy_install -U pyRserve). If you are at EuroPython, too, and want to talk about it just come and see me.

About pyRserve

pyRserve is a (pure python) client for connecting Python to an R process on a remote server via TCP-IP (using Rserve). R is one of the most important and most widely spread open source statistic packages available.

Through such a pyRserve connection the full power of R is available on the Python side without programming in R. From Python variables can get set in and retrieved from R, and R-functions can be called remotely. No R-related libraries need to be installed on the client side.

Sample code

>>> import pyRserve
>>> conn = pyRserve.connect('servername.domain.com')
>>> conn.r('1+1')                # direct evaluation of a statement in R
2
>>> conn.r.myList = [1, 2, 3]    # bind a Python list in R to variable 'myList'

>>> conn.r('func1 <- function(v) { v*2 }') # create a function in R
>>> conn.r.func1(4)                        # call the function in R
16

Most important changes in V 0.6.0

  • Support for Python 3.x (therefore dropped support for Python <= 2.5)
  • Support for unicode strings
  • Suport for Fortran-style ordering of numpy arrays
  • Elements of single-item arrays are now translated to native python data types
  • Full support complex numbers, partial support for 64bit integers and arrays

Documentation and Support

The documentation for pyRserve is available at http://packages.python.org/pyRserve

The corresponding Google news group can be found at http://groups.google.com/group/pyrserve

Read and Post Comments

pyRserve 0.5.2 released

December 06, 2011 at 08:22 PM | categories: Python, R | View Comments

The latest version is now available via pypi (easy_install -U pyRserve).

About pyRserve

pyRserve is a (pure python) client for connecting Python to an R process on a remote server via TCP-IP (using Rserve). Through such a connection variables can be get and set in R from Python, and also R-functions can be called remotely. No R-related libraries need to be installed on the client side.

Sample code

>>> from pyRserve import connect
>>> conn = connect('your R server')
>>> conn.r('1+1')                # direct evaluation of a statement
2
>>> conn.r.myList = [1, 2, 3]    # bind a list within R to variable 'myList'

>>> conn.r('func1 <- function(v) { v*2 }') # create a function in R
>>> conn.r.func1(4)                        # call the function in R
16

Changes in V 0.5.2

  • Fixed bug with 32bit integer conversion on 64bit machines. Upgrade highly recommended!

Documentation and Support

The documentation for pyRserve is available at http://packages.python.org/pyRserve

The corresponding Google news group can be found at http://groups.google.com/group/pyrserve

Read and Post Comments

Next Page ยป