Friday, June 28, 2013

Adding external libraries (jar files) to an ivy project (Nutch)

I wanted to use my Apache Giraph fork in Apache Nutch 2.1. The Problem was that Giraph used mvn and Nutch used ivy. Although Nutch could fetch jar files from maven repositories, my customized jars are in none of those repos. So I had to find a way to use my local jars.

Add the following line to ivysettings.xml of Nutch:
<module organisation="org.apache.giraph" name=".*" resolver="internal"/>

Add the following lines to ivy.xml of Nutch:
    <dependency org="org.apache.giraph" name="giraph" rev="1.1.0-SNAPSHOT"
      conf="*->default" />
    <dependency org="org.apache.giraph" name="giraph-hbase" rev="1.1.0-SNAPSHOT"
      conf="*->default" />
    <dependency org="org.apache.giraph" name="giraph-examples" rev="1.1.0-SNAPSHOT"
      conf="*->default" />

Now create put jar files under .ivy2 subfolders:


Now when you run ant runtime and start a crawling, you will see that your local jar files are binded to your app:
SLF4J: Found binding in [jar:file:/home/emre/tmp/apache-nutch-2.1/runtime/local/lib/giraph-1.1.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/emre/tmp/apache-nutch-2.1/runtime/local/lib/giraph-examples-1.1.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/emre/tmp/apache-nutch-2.1/runtime/local/lib/giraph-hbase-1.1.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]

Thursday, June 20, 2013

Proxy Pattern for Extending non-extendable Python Class

graph_tool library is based on boost C++ library and provides Vertex class binding for Python. If we wanted to extend this Vertex class and add some attributes and methods, it wouldn't let us do that due to private constructor in C++ code.

RuntimeError: This class cannot be instantiated from Python

We can overcome this obstacle using Proxy pattern.

In the __getattr__ method, if the attribute(or function name) is not in the Subclass MyVertex, then it looks for attributes of Vertex object that is defined inside MyVertex.
 Here is the code:

Saturday, June 8, 2013

Latex and Bibtex

Using bibliography in latex might be headache sometimes. While I was writing my MSc thesis, I had taken some notes. I'd like to share them with you.

For a while, I was wondering what is the difference between bib and bbl. I learned that when you execute bibtex, bib files are converted to bbl.

Steps for running BIBTEX with LaTEX

1. Run LaTEX, which generates a list of \cite references in its auxiliary file, .aux.
2. Run BIBTEX, which reads the auxiliary file, looks up the references in a database
(one or more .bib files, and then writes a file (the .bbl file) containing the formatted
references according to the format specified in the style file (the .bst file). Warning
and error messages are written to the log file (the .blg file). It should be noted that
BIBTEX never reads the original LaTEX source file.
3. Run LaTEX again, which now reads the .bbl reference file.
4. Run LaTEX a third time, resolving all references.