Ghostwritten Insomnia

Postgres deadlock debugging

2011-01-19T10:35:00.000-08:00

Found a nice SQL query for debugging deadlocks hidden deep in the postgres docs:

select
  pg_stat_activity.datname,
  pg_class.relname,
  pg_locks.transactionid, 
  pg_locks.mode,
  pg_locks.granted,
  pg_stat_activity.usename,
  substr(pg_stat_activity.current_query,1,40) as current_query, 
  pg_stat_activity.query_start, 
  age(now(),pg_stat_activity.query_start) as "age", 
  pg_stat_activity.procpid 
from pg_stat_activity,pg_locks 
left outer join pg_class on (pg_locks.relation = pg_class.oid)  
where pg_locks.pid=pg_stat_activity.procpid order by query_start;

Cyberpunk coding fonts - monofur on Linux

2010-08-10T10:07:00.000-07:00

If you ever grow weary of Inconsolata, try monofur. Adding Truetype fonts to Linux is a piece of cake with fontypython. Just install it via Synaptic or apt-get.

Danmark

2010-08-08T05:15:00.001-07:00

La Joconde

2010-05-11T14:29:00.000-07:00

How to compile Vim on Linux

2010-05-05T10:25:00.000-07:00

http://users.skynet.be/antoine.mechelynck/vim/compunix.htm

Klartext

2010-04-17T11:20:00.000-07:00

SQL for removing invalid foreign keys and typecasting in Postgres

2010-04-16T05:44:00.000-07:00


DELETE FROM session_data WHERE session_data.name = 'user_id' 
AND NOT EXISTS 
(SELECT * FROM users WHERE CAST(session_data.value AS integer) = users.id);



Debugging nosetests with ipdb
2010-01-08T06:08:00.001-08:00

Ever wanted to debug Python with tab completion and syntax highlighting? Then you'll love ipdb:

sudo easy_install ipdb


The only thing you have to do is use ipdb instead of pdb:

import ipdb; ipdb.set_trace()


With a little trick it will even work with nosetests:

import sys; sys.stdout = sys.__stdout__; import ipdb; ipdb.set_trace()



Wanda's Wisdom
2009-09-14T16:04:00.000-07:00
You may be gone tomorrow, but that doesn't mean that you weren't here today.



Break the Cycle: Local Class Definitions
2009-09-07T08:29:00.000-07:00
In the process of analyzing the memory behavior of your Python application, you will sooner or later stumble across reference cycles. It is always a good idea to avoid creating reference cycles, though not every cycle is worth breaking (using weak methods avoided some reference cycles but increased the invocation cost).

Debugging reference cycles can be simplified by creating a graphical representation of the reference graph, e.g. using graphviz. Marius Gedminas provides a set of tools to facilitate building graphs at his homepage. Similar facilities exist in Pympler. The latter improved considerably since the official 0.1 release so be sure to grab the version from the svn trunk.

When you know what objects are involved in cyclic dependencies you will want to know why these occurred in the first place, which is not always trivial to figure out. While working on the integration of Bottle (which is really great BTW) in Pympler, I stumbled across an interesting case:

import gc
gc.disable()

def f():
    class Foo(object):
        pass
f()

from pympler.gui.garbage import GarbageGraph
GarbageGraph(reduce=True).render('cycle1.png', format='png')


This snippet creates the following reference cycle (click on the image to enlarge):



Apparently, defining a class in the local scope of a function or method creates a reference cycle. Lifting the class definition to the module level avoids the reference cycle. It is even more interesting that class objects create reference cycles by design when they go out of scope:

>>> import gc
>>> gc.disable()
>>> class Foo(object):
...   pass
>>> del Foo
>>> gc.collect()
6


So what? Well, it is evidently beneficial to define classes in modules or other classes, and not in functions or methods.


Substitute assert statements with unittest methods using vim
2009-09-01T03:54:00.000-07:00
In the Python community, it's not generally agreed upon whether to use the assert statement or the assert* methods from the unittest module. As some commentators pointed out in a recent discussion, there are (a few) good reasons to prefer the assertion methods, e.g. better error messages.

Here are some vim substitution commands that make the transition from assert statements to the appropriate methods easier:

:%s/assert \(.\+\) == \(.\+\)/self.assertEqual(\1, \2)/gc
:%s/assert \(.\+\) != \(.\+\)/self.assertNotEqual(\1, \2)/gc
:%s/assert \(.\+\)/self.assert_(\1)/gc



Convert Images to A4 PDF
2009-08-19T12:07:00.000-07:00
Converting raster images to PDF in a printable format can be achieved using the ImageMagick convert utility with the page parameter:

convert -page a4 *.png images.pdf


The converter, however, not quite does what I expected. Images are resized to fill the A4 page but the aspect ratio is preserved and no margin is added. This actually leads to different sized pages for images with different ratios (which is common for scanned documents for example). 

In order to create equal-sized PDF pages from a bunch of images, a margin or border needs to be added to the images. Doing this manually is a cumbersome process. Therefore, I've written a little Python script which adds a (white) border to the individual images to enforce an aspect ratio compatible with A4 pages. The script creates a PDF file from a bunch of image files with uniform A4 page size:

import sys
from subprocess import Popen, PIPE

PAGE_WIDTH = 210.0
PAGE_HEIGHT = 297.0

files = [arg for arg in sys.argv[1:-1]]
output = sys.argv[-1]
tmp = ["a4%s" % f for f in files]
for f,t in zip(files, tmp):
    p = Popen(["identify", f], stdout=PIPE)
    dim = p.communicate()[0].split()[2]
    w,h = [float(d) for d in dim.split('x')]
    bw,bh = 0,0
    if w/h < PAGE_WIDTH/PAGE_HEIGHT:
        nw = PAGE_WIDTH * h / PAGE_HEIGHT
        bw = int((nw - w) / 2)
    else:
        nh = PAGE_HEIGHT * w / PAGE_WIDTH
        bh = int((nh - h) / 2)
    Popen(["convert", "-border", "%dx%d" % (bw,bh), 
        "-bordercolor", "white", f, t]).communicate() 
Popen(["convert", "-page", "a4"] + tmp + [output]).communicate()


Save the script to img2a4pdf.py and invoke it like that:

python img2a4pdf.py *.png output.pdf


Maybe someone will find it useful.


Der neueste Kick
2009-07-22T12:06:00.000-07:00
Mal wieder eine Meldung auf Tagesschau.de:

"Es ist offensichtlich, dass unsere Milchbauern gerade leiden", sagte sie. "Es geht um echte Menschen und nicht um Statistiken auf einem Blatt Papier."


Etwas weiter unten dann:

Der deutsche Bauernverband hatte das Schlachten von Kühen als eine Möglichkeit gesehen, um das Überangebot an Milch auf dem europäischen Markt zu bereinigen: Wenn anderthalb Millionen Tiere getötet würden, könnte das den Kick geben, um aus dem tiefen Tal wieder herauszukommen.


Mit etwas Glück kommt vielleicht bald die Milch im Kaffee nach dem Steak vom selben Tier. Prost Mahlzeit!

Immerhin, das Fleisch von anderthalb Millionen geschlachteten Rindern reicht in Deutschland keine sechs Monate. Wer wäre da nicht gern Vegetarier.


Inkscape PDF Export
2009-07-19T10:17:00.000-07:00
Inkscape is nice vector drawing program, especially for illustrating Latex documents. In the past, I've always exported to eps first and then converted the docs to PDF using epstopdf. This way, the bounding box is confined to the actual region of interest.

Unfortunately, transparency information is lost in the process. What works, though, is to fit the page to the selection just before directly exporting to PDF. Go to File > Document Properties and press Fit page to selection on the Page tab.


Almost done
2009-07-17T11:04:00.000-07:00
At last, I've completed my final thesis. Seven exciting years as a student passed almost too quickly.  In a week from now, it'll all be over. Finally, time for traveling, working on Pympler and SCons, getting an interesting job.

In the process of finding a new home, our old server will probably be disconnected before very long.  Therefore, it is time to find a new haven for ideas, thoughts and casual code snippets.


Empirical Comparison of SCons and GNU Make
2008-08-31T14:39:00.000-07:00
A course work of mine, entitled Empirical Comparison of SCons and GNU Make, is now hosted at the company site of my tutors. Thanks guys!


Sverige
2008-08-13T14:02:00.000-07:00



Upgrade to Hardy Heron vs Trac and sqlite
2008-04-10T12:22:00.000-07:00
Today I tried something which I don't really expected to work. After a direct upgrade from Dapper to Hardy, Trac stopped working:

"file is encrypted or is not a database"


There's a version conflict between the Trac database and the accessing sqlite version. The best solution I found while searching the web, was to convert the databases manually:

sudo -s
apt-get install sqlite
cd /var/your-trac-project/db
cp trac.db trac.db.orig
sqlite trac.db .dump > trac.sql
rm trac.db
sqlite3 trac.db ".read trac.sql"
chown www-data:www-data *
trac-admin .. upgrade


After that, Trac worked again.


Automatic make dependencies without sed
2008-03-01T12:17:00.000-08:00
I'm currently working with plain GNU Make based projects and - again - I tried to find a nice way to automatically track dependencies and finally found a nice solution when skimming over the g++ manpage. There is a lot of black art flowing around, e.g. this gem is from the GNU Make manual:
%.d: %.c
   @set -e; rm -f $@; \
    $(CC) -M $(CPPFLAGS) $< > $@.$$$$; \
    sed 's,\($*\)\.o[ :]*,\1.o $@ : ,g' < $@.$$$$ > $@; \
    rm -f $@.$$$$


Not for the faint of heart, also the thing done by sed is quite trivial. If all you have to support is gcc (portability is overrated!), there's a much more simple way to achieve the same (and more):
%.o: %.c
   $(CC) -MMD -MP -MT '$*.d' -c $(CFLAGS) $< -o $@



Hrvatska
2007-09-15T14:26:00.000-07:00



Norge
2007-08-05T10:47:00.000-07:00