POX and Gephi graph streaming / visualization

June 2, 2013 in News, POX

POX has an unsupported visualization GUI in the form of POXDesk, but today it got another somewhat different option. Gephi is a great (open source) graph visualization and manipulation package. It has a plugin for streaming graphs in and out over HTTP, and POX’s new misc.gephi_topo component uses this to stream switches, links, and (optionally) hosts to it. Once the graph is in Gephi, you can export it to graph file formats, analyze it in a bunch of different ways, just look at it, render it as SVG, and so on. Read on for more info and a screenshot.

Although the code is based on POXDesk, the ability to view hosts is new (it’d be nice to port it back to POXDesk, though!). Adding them in required a bit of improvement to the host_tracker component — specifically adding a HostEvent. host_tracker could still use some work to get it a little more consistent with changes in POX that have happened over the last couple of years, but it’s cool that it still works, and with the addition of events, it should be more useful and easier to use.

This also serves as an example of creating arbitrary socket servers using IOWorker. In theory, Gephi graph streaming is HTTP, but the POX component currently just streams the graph blindly to anything that connects.

gephi_topo

18 responses to POX and Gephi graph streaming / visualization

  1. Hi,
    I have the similar problem
    INFO:misc.gephi_topo:Client connect
    INFO:misc.gephi_topo:Client disconnect

    java version “1.7.0_51″
    gephi: 0.8.2

    i am new to gephi can you help me out?

  2. Thanks for the help. I’ve pushed the code to dart.

    And after another look at pygephi_graphstreaming, I definitely had seen it before (the Twitter example reminded me). For the moment, though, I’m sticking with my arguments for text/plain over application/json as given above.

  3. I’ve followed the instructions, but the Gephi fails to connect to POX. More precisely, Gephi connects to the controller with the following request:

    GET / HTTP/1.1
    Cache-Control: no-cache
    Pragma: no-cache
    User-Agent: Java/1.7.0_17
    Host: localhost:8282
    Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
    Connection: keep-alive

    But then the connection is closed and I have a red light besides “http://127.0.0.1:8282″ in Gephi. However, the gephi_topo.py seems to work, because when I telnet to POX, I get:

    {“dn”: {“filter”: “ALL”}}
    {“an”: {“00-00-00-00-00-01″: {“kind”: “switch”, “label”: “00-00-00-00-00-01″}}}
    {“an”: {“00-00-00-00-00-02″: {“kind”: “switch”, “label”: “00-00-00-00-00-02″}}}
    {“an”: {“00-00-00-00-00-03″: {“kind”: “switch”, “label”: “00-00-00-00-00-03″}}}
    {“ae”: {“00-00-00-00-00-01_00-00-00-00-00-02″: {“directed”: false, “source”: “00-00-00-00-00-01″, “target”: “00-00-00-00-00-02″}}}
    {“ae”: {“00-00-00-00-00-02_00-00-00-00-00-03″: {“directed”: false, “source”: “00-00-00-00-00-02″, “target”: “00-00-00-00-00-03″}}}

    Do you know what’s going on? I use Gephi 0.8.2 201210100934 with Graph Streaming Version 0.8.1.0

    Thanks,
    Felicián

    • Hopefully we can sort it out. :)

      I’m using the same versions of Gephi and Graph Streaming as you are, and things are working for me, so that wouldn’t seem to be it.

      Our Java versions are different, so that’s one possibility. The Gephi streaming component actually totally ignores HTTP and just sends the data. With my version of Java this works, but it’s possible yours is more picky.

      Another possibility is platform. Are you on Windows, by any chance? In misc/gephi_topo.py or whatever, there are a couple places where it constructs strings using \r\n… if you replace these with \n, do things work?

      • Oh, also… is there a possibility that the graph is modified but not visible? You can probably check this by switching to the Data Laboratory.

        • You are right about Windows. I started POX in a VM, and tried to connect Gephi to it from a Windows host.

          But today I downloaded the mininet 2.1.0 VM (64 bit) and installed guest-additions, default-jre, xinit, x11-xserver-utils, fluxbox, and Gephi on it. Then:

          $ sudo -E mn –topo=linear,k=3 –remote

          $ ./pox.py forwarding.l2_learning misc.gephi_topo openflow.discovery

          But it fails the same way as previously. Besides carp, I tried the dart branch as well.

          (I’m totally new to Gephi, so it’s possible the I’m missing something completely trivial, but Data Laboratory is empty as well, and on Overview the Context window shows: “Nodes; 0, Edges:0″. Additionally, I think I should see a green light before “http://127.0.0.1:8282″ similarly to your screenshot, but it’s red.)

          Thank you!

          • The following patch fixes this for me. Shall I send a pull request?

            diff –git a/pox/misc/gephi_topo.py b/pox/misc/gephi_topo.py
            index 5c82933..81d34c5 100644
            — a/pox/misc/gephi_topo.py
            +++ b/pox/misc/gephi_topo.py
            @@ -125,8 +125,8 @@ class GephiTopo (object):
            out.append(ae(h,s))

            out = ‘\r\n’.join(json.dumps(o) for o in out)
            -
            - client.send(out + ‘\r\n’)
            + header = ‘HTTP/1.1 200 OK\r\nContent-Type: application/json\r\n\r\n’
            + client.send(header + out + ‘\r\n’)

            def __handle_host_tracker_HostEvent (self, event):
            # Name is intentionally mangled to keep listen_to_dependencies away

          • (Excuse the out-of-order reply… apparently they only nest just so deep.)

            Thanks for looking further into this! I actually had some code written to parse the HTTP and stuff a little bit, but never really bothered to finish it since it worked fine without it (at least for me!). I took a minute to hack it enough to work, refactoring the whole thing to make it slightly more generic in case we want to get the “raw” behavior or something back in the future.

            Do you want to give it a test? If it works, I’ll push it (and credit you for helping me track this down!).

            (URL removed; this code is now in the dart branch.)

          • Yes, your new code does work for me. However, it replies with “test/plain’, but sample codes from https://github.com/panisson/pygephi_graphstreaming replies with “application/json”.

            BTW, POX-Gephi integration seems really cool. Lots of new possibilities… For example, hacking something around the GeoLayout plugin is really tempting.

          • I don’t think I’d seen pygephi_graphstreaming before, but the reason I used text/plain is because that’s what the Gephi plugin does when it’s running in server mode. I think the output may not actually be proper application/json, since I think application/json responses should be a single JSON object, whereas Gephi actually sends many (framed by new lines, I believe).

            Thanks for testing, and the GeoLayout idea is a cute one.

            By the way, it was great to see someone make serious use of (and extensions to) POXDesk! I thought it looked really good. :)

Leave a reply

You must be logged in to post a comment.

Sign in using...