Thursday, December 16, 2010

ParallelPython vs multiprocessing

Today I'm working on parallelisation of a process I have written. Process is simply a text conversion. I have a translator class, organism directories and files inside these directories. What I want to do is to split the data among the processors and perform the operation faster.

To do this, I tested multiprocessing and ParallelPython modules of python. Without using these modules, it took 38 seconds to perform the task whereas with the help of these modules, it went down to 29 seconds(multiprocessing) and 30 seconds (ParallelPython). Not a great deal but better than nothing. By the way, ParallelPython is way too complicated compared to multiprocessing.

Here is the code for ParallelPython:

import translator
import os
from utils.pp import pp

base = "/some/path"
organisms = [ "organism1", "organism2", ...]

def convert_organism(base, organism):
t = translator.BiogridOspreyTranslator()
# uses os module here
t.translate()

if __name__ == '__main__':
job_server = pp.Server(ppservers=())
jobs = [(organism, job_server.submit(convert_organism, (base, organism,), (), ("os","translator",))) for organism in organisms]
for organism, job in jobs:
job()




ParallelPython requires you to tell him the modules the functions requires. I didn't like that.
And here is the code for multiprocessing:

from translator import *
import os
from multiprocessing import Pool

base = "/some/path"
organisms = [ "organism1", "organism2", ...]

def convert_organism(organism):
t = BiogridOspreyTranslator()
# uses os module here
t.translate()

if __name__ == '__main__':
pool = Pool(processes = 2)
pool.map(convert_organism, organisms)

Tuesday, December 7, 2010

Uploading on sourceforge

Today, we're trying to publish our project Robinviz. It'll be a 1.0-beta version seperately for windows/linux source/binary. Binary files are around 400MB so it's a bit hard to upload it. Google Code -which we use for project management- limits the maximum file size for uploads. So we had to switch to sourceforge. But uploading on http was still a problem. So I found this solution: Rsync over SSH

At first, I found it hard to understand the URL format but then understood and wanted to share it with you. With the following command, I was able to send my 400MB file with resume support:

rsync -avP -e ssh robinviz-1.0-beta-linux-binary.tar.gz aladagemre,robinviz@frs.sourceforge.net:/home/frs/project/r/ro/robinviz/linux-1.0-beta

robinviz-1.0-beta-linux-binary.tar.gz: filename
aladagemre: username on sourceforge
robinviz: project name
/r/ro/robinviz: the first letter, the first two letters and all letters of the project joined by /
linux-1.0-beta: the folder I'd like to put my file in.

Hope it's useful.

Saturday, October 30, 2010

Dell Wireless 1515 Wireless-N Adapter and WPA timeout on Vista

I'm using a Dell Wireless 1515 Wireless-N Adapter and having connection problems in Vista. When I use WPA for authentication, it doesn't connect to my wireless network and says authentication failed due to timeout. Sometimes it doesn't connect and very rarely it connects. When I looked at the Event viewer, I saw the following details:

Detailed root cause:
Layer 2 security key exchange did not generate multicast keys before timeout


After a long research, I thought that I found a way to fix that. I went to Dell drivers site and wrote my laptop Service tag to get appropriate driver updates. I downloaded the one for my wireless adapter: R198377.exe. The reason why the research was long is that I had to do it on another OS and wanted to find a sound solution not to fail and return back. The dates for the driver were earlier than the date I purchased my laptop but it seems like the drivers coming with it are pretty old.

After installing that, my wireless card -surprisingly- immediately connected to the wireless network. I was gonna say "Well done!" but it turned out to be a coincidence that it worked! After a restart, it was the same problem again. Re-installing driver didn't work. Sigh.

I don't know what to do with this.

Sunday, September 5, 2010

Keyword Argument Injection with Python Decorators

In most of the object oriented codes we write, we need to set class variables to the given argument values and this is a very line-consuming thing. See what I mean:

class Test:
def __init__(self, x, y, z, t):
self.x = x
self.y = y
self.z = z
self.t = t
print x,y,z,t


To get over these redundant lines, I found a solution using decorators:

def injectArguments(inFunction):
def outFunction(*args,**kwargs):
_self = args[0]
_self.__dict__.update(kwargs)
inFunction(*args,**kwargs)
return outFunction

class Test:
@injectArguments
def __init__(self, x, y, z, t):
print self.x,self.y,self.z,self.t

@injectArguments
def fonksiyon(self, ad):
print "Ad:",self.ad

t = Test(x=4, y=5, z=6, t=7)
t.fonksiyon(ad="Emre")


args dictionary contains the arguments and kwargs dictionary contains the keyword arguments. arg[0] corresponds to self argument. So what we do is we update self.__dict__ with {'x':4, 'y':5,'z':6,'t':7} where self corresponds to the Test instance. This way, bulky self.x = x, etc. codes are eliminated.

I don't know if any built-it decorator like this exists in Python libraries but it seems like I'll be using this a lot.

Thursday, July 8, 2010

QActionGroup and exclusive menu actions

Today I was working on exclusive menu action selection in PyQt4 and got stuck. After some struggling, I managed to find it out. Now I want to share it with you.

What is exclusive menu action? It's a menu item, when you select it, its friends are deselected. When one of its friends is selected, it gets deselected. It's like a radio button group in a menu.

We're going to use QMenu, QAction and QActionGroup for this purpose.
# assume we are in a QMainWindow method, constructing the menu.

menu = QMenu('Alignment', self.menuBar())
menuGroup = QActionGroup(menu)
menuGroup.setExclusive(True)

left = QAction('left', menuGroup)
center = QAction('center', menuGroup)
right = QAction('right', menuGroup)

actions = (left, center, right)
map(menu.addAction, actions)
map(lambda action: action.setCheckable(True), actions)

center.setChecked(True)
self.menuBar().addMenu(menu)


In this way, you are able to select either left or center or right for alignment submenu.

Tuesday, June 29, 2010

QGraphicsItem.itemChange event and ItemPositionChange in Qt 4.6

Recently, I had a bug in Robinviz that I could not figure out where it come from. I was drawing a graph and nodes could be moved. Whenever I moved the node, edges connected to it used to come with it to the new position. But after upgrading to Qt 4.6, I saw that edges were not moving.

After some investigation, I realized that QGraphicsItem.itemChange event did not produce QGraphicsItem.ItemPositionChange but only selected, deselected signals. Googling for it, I hardly found that the problem was with the update. Due to performance reasons, Qt developers decided to stop emitting geometrical signals and wanted us to switch it on by supplying a flag for the QGraphicsItem:


try:
# available only in Qt 4.6
self.setFlag( QGraphicsItem.ItemSendsGeometryChanges)
except:
# no need to do this in Qt 4.5
pass

You can do this flag option in your constructor. I used try/except because I didn't know what might happen in 4.5 as there was no flag called ItemSendsGeometryChanges. In some other websites, following flag was suggested but it did not work for me:

self.setFlag(QGraphicsItem.ItemSendsScenePositionChanges, True)


For those who might be interested how I used itemChanged, I'll provide a portion of my code:

def itemChange(self, change, value):
if change == QGraphicsItem.ItemPositionChange:
self.updateEdges()

return QVariant(value)

Saturday, May 29, 2010

Spring term finished

The spring term has finished. Courses were pretty time-consuming this semester so I could not focus o n our research as I did last semester. Grades are not available yet but I expect them to be satisfying. I've taken Data Mining, Analysis and Design of Algorithms, Software Engineering courses.

In the data mining project, I worked on Yeast localization of proteins database and did classification&clustering analysis.

In the Software Engineering project, as I told before, I've developed a bus-timetable program JIstanbul which has both mobile and desktop versions with J2SE+Swing and J2ME. My project was selected as the best in the class. Currently, in the mobile version, the program can detect the day and fetch the timetable for that day. In the future, I'm planning to add storing time information, closest bus time, frequently used buses features.

Now I have 1 course and masters-thesis left for the next year. I'll be focusing on Robinviz in these summer days.

Sunday, April 4, 2010

RobinViz Paper and School Projects

After releasing RobinViz, I focused on my school projects. I've written a Turkish Flag drawer, Conway's Game of Life and Julia Set Fractals drawer in Java and presented the latter one in the Software Engineering class.

Currently I'm developing an Optimal Binary Search Tree implementation in Python as part of my Algorithms class. I've registered for a github account and I put all my codes there.
On the other hand, I'm working on some edge bundling stuff nowadays as part of my academic work. It's really nice but complicated. I hope I can come by a good solution to our problems in our mind.

I think I'll be fixing some problems of Robinviz in the following months. Our published paper (An integrated model for visualizing biclusters from gene expression data and PPI networks) can be found here.

As part of my Software Engineering project, I'm planning to develop a J2ME application to fetch bus timetables from the Internet and display the closest bus times for you to find the best route without waiting your bus.

Thursday, February 18, 2010

RobinViz Beta released

Things are going well and I've finished my first semester of my Masters. All of my grades are great and I'm just starting the second semester. On the other hand, I'm working on our project. When I joined the project in September 2009, one last step was missing. That was the Horizontal Coordinate Assignment Problem in Layered Graph Drawings. I've implemented Fast and Simple Horizontal Coordinate Assignment paper of Brandes et. al and we submitted our paper to International Symposium in Biocomputing (ISB 2010) in Calicut, India. Our paper was accepted and after that we decided to improve the GUI of our implementation. We developed a brand new GUI with PyQt4 and used the C++ code behind the scene to do the most of the scientific calculations. We published this novel software, RobinViz Beta version on http://code.google.com/p/robinviz.

With RobinViz, you can visualize PPI Networks and Gene Ontology as biclustered graphs. In these graphs, reliability of interactions are expressed with the thickness of the edges, h-value of the biclusters are expressed as node width etc. So the most important part of the data comes in front. I hope it shall be useful for all the scientists working on this subject.

This semester I'll be taking Algorithms, Data Mining and Advanced Java courses. I hope I'll learn a lot from them. By the way, while our professor presented our paper at the conference, I gave Introduction to Java tutorial lectures in his classes. That was also a nice experience.

Tuesday, January 5, 2010

Avatar 3D

After a long time, I went to cinema, to the film Avatar. I was not sure wheter to see this film but I wanted to taste a 3D movie so this could be a good candidate for my first 3D film.

The film was more than I expected, but not the 3D effects. It had both sorrow and thrill. For the ones who was not seen it, I won't spoil it but here's a summary.

It's the year 2154 and humans want to mine a valueable mineral on the Planet Pandora. But this mineral lies under a great tree, in which all the Pandora folk lives. So humans want Pandorians to leave their homes but of course, Pandorians won't do it. So a war begins and one human, who controls a mind-controlled avatar (Pandorian body) for diplomatic purposes, sees the fact that this is not fair and lines up on the Pandorians side. Going further, he becomes the commander of them by achieving the rank "Toruk Macto".

I felt so ashamed of being a human while watching this film. How cruel these humans are!