Firefox 3 is out! Grab it from http://www.mozilla.com/firefox/.
One thing that I noticed is that at times the site will take a long time to respond. I suppose all the buzz about Firefox is really hitting their servers hard.
Initial impressions: much faster, much more light-weight than Firefox 2. I would definately recommend anyone with a lower end machine to upgrade. Downside: finding all my favourite Firefox extensions and plugins that are compatible with Firefox 3. The search begins!
With a risk of starting a religious war between these schools, I will be bold enough to say that when it comes to Java development, IntelliJ wins hands down, and I’ll explain my perspective below.
But, before moving on to the gist of it, let me add some context and background to my current situation. Having been a geek at heart, I am always looking for even geekier ways of doing things. For example, rather than just deploying one of the few hundred blogging applications out there, I’ve written my own using Django. It’s not because I have a masochist mind, but because I find it fun, exciting and challenging. Based on that, I can basically summarise my professional (IT and programming) needs as: geeky, fun and exciting. Others, come second (yes, that includes remuneration and benefits). I’ll jump on to anything that is interesting and challenging. With that out of the way, lets continue.
I’ve been building front office trading applications for a good part of 2.5 years now (straight out of college). All of it in Java. No J2EE, just all in good ol’ plain J2SE. In a short span of time, I’ve managed to use so many Java technologies – Hibernate, Jetty, Tomcat, JBoss (we never used it as a J2EE container), Log4j, Slf4j, commons-*, etc. I’ve learnt how to write performant code, to think twice on how a JVM might compile this code, and how JIT might enhance it. I’ve learnt to plan and code very quickly as well. Summing all of these up, I need an excellent tool for programming to ensure that I spend all my time solving the problem at hand, rather than solving the problems of the tool.
I had used Eclipse in college, albeit sparingly. It was OK, and I thought that IBM did do a great job in initiating it. Then I graduated and started at my first job. The team had a standard on using IntelliJ, and for the first time in my life, I started using it. I saw how team members around me were using it and I picked up some shortcuts and tips on how to code quickly by leveraging IntelliJ features. At some point I saw this one guy use IntelliJ and he was literally killing it. Till that point, I have never seen so much code being generated in such a short time, and it was useful code! Not templated rubbish and stuff that you’d add to an API “because someone might need it in the future”. He was really productive. So, I did what I think has changed the way that I code: I copied him (shamelessly)! I observed and learnt what he was doing, and why. How his thoughts flowed while coding and how he tied that in with his tool – IntelliJ. Safe to say – I learnt more about IT and programming in my first 4 months of my job than 4 years in college.
From then on, I did not look back on Eclipse at all for Java. I even bought a personal license for IntelliJ. It was one of the most productive times in my professional career. I took ownership of the team sometime early 2007, and was advocating IntelliJ to everyone in the firm. Everyone got about doing their work, and maybe once in a while had a tiff with IntelliJ, but that’s about it. Best part of all was that it was fun coding with IntelliJ. It does what I’m expecting it to do, and quickly at that. Refactoring is a sheer pleasure as well.
Towards the end of last year, I had a bit of a problem. One of my servers had 2GB of physical RAM, and I needed to chuck in 10-20 JVMs for one of my personal projects, and that just won’t scale well. FYI, a busy running JVM does take quite a lot of heap space. So, I started looking at Python as a light weight alternative. I really like Python as a language, and am advocating its use in building low latency trading apps (tied in with the use of Psyco). The only problem was that there wasn’t an IntelliJ equivalent for Python. All the IDEs just sucked! I wanted to be able to kill an IDE and spew out just as much code as I did when programming Java with IntelliJ.
This was when I started using Eclipse heavily. I settled with PyDev with Extensions and it’s actually not too bad compared to the other Python IDEs out there. Take away all Eclipse’s nuisance, the plugin is pretty polished. It has code completion, some refactoring, satisfactory code browsing and as a whole it was OK. I started to convert most of my Java personal projects to Python, though not at the level of productivity that I had desired. IntelliJ 8 is due to have good Python support (including debugging). If that’s satisfactory, I’m ditching Eclipse for my personal
At this point, I was using IntelliJ at work, though I did not code much for a year. I did build prototypes, but most of my time was spent in managing the team and running the show. I used Eclipse outside work for Python. In effect, I ended up spending most of my coding time in Eclipse.
At the beginning of this year, I moved firms and am back to coding full time. At this new place, the mandate is to use Eclipse. What a shocker it was! My productivity almost halved. What took me 30 minutes to complete in IntelliJ took an hour in Eclipse, due to its clunky refactoring, code browsing, searching and interface. At this point I realised what the problem with Eclipse was: its just trying to satisfy too many people. It’s trying to be the mother of all IDEs and support every bloody language in the world. In my opinion, IntelliJ have perfected this – for a long time, they’ve focused primarily on Java. They’ve done an amazing job, and at the moment, just replicating their success to other languages. Being unhappy with coding Java in Eclipse, I went about in the firm asking why there aren’t many IntelliJ users – the main reason: cost! Apparently, the cost to benefit ratio of IntelliJ was appaling.
I am not entirely sure how they conducted their research, but in my humble opinion, there is almost no way that you can make a statement like that. While IntelliJ costs money, you quickly recover that cost from the individuals higher contribution levels. In the big picture, its an extremely small cost ($599 per enterprise license) compared with the other costs of keeping a developer seated in an office. You have overheads, admin costs and other misc things to worry about, which really makes the one off $599 miniscule. In fact, I think Eclipse actually costs more – the time spent working with the tool, instead of the problem at hand has an internal cost. There is no way in Eclipse to get about doing a big refactor of a few code bases without changing perspectives (most common for me is switching between Java and Team Synchronisation). The lower levels of productivity have more costs associated to it. What about repercussions of delivering late? Reputational risk? I can just carry on and on about the extra costs that comes with Eclipse. Plenty of folks in management who decide on Eclipse over IntelliJ just don’t realise that the $0 on Eclipse is not a true $0. At the same time, the $599 for IntelliJ is not a true $599 – there are additional costs with using IntelliJ as well, but in the big picture in hiring and keeping a developer in office, its amounts to something really small. The benefits from using IntelliJ are pretty evident – it’s so much simpler to use, and developers tend to spend more time on the problems at hand rather than on the IDE itself. How did they carry our the cost-benefit analysis? I can only assume that they took the cost of Eclipse at a flat $0, which really does not reflect reality.
So, here I am. For the past 4 months, during my day job I am using Eclipse for Java and at home using Eclipse for Python, until this morning when I added a tutorial section to XP-Dev.com (which is still in Java using the amazing Wicket framework) and fired up IntelliJ for the first time. I added some code here and there, and only took a few minutes to add the section in and deploy it out. I was so relieved and excited to use IntelliJ, that I just had to rant about it and how Eclipse has been picked as a standard Java IDE in many organisations due to costs.
I am waiting eagerly for Python support to improve in IntelliJ, and then I can start using it again for my personal projects. On the work front – I never give up without a fight. I will carry on advocating IntelliJ for the right reasons, even if it’s all about the costs. I do want to put back the fun in coding at the workplace! I do want to get the firm (or at least my immediate team members) on to higher levels of productivity.
I’ve had this script lingering around for a bit to convert almost any video file that MPlayer can read to one that can be played on a Sony Playstation Portable (PSP).
First, you have to ensure that you’ve got the latest PSP system software (I’ve got version 3.72 installed, at time of blogging, version 3.90 is the latest one).
Then, install mplayer and mencoder. If you’re using Ubuntu, this is as simple as:
$ sudo apt-get install mplayer mencoder
Finally, copy and paste the following script to a file. Lets call the file ‘convert_psp’:
#!/bin/bash
if [ -z "$1" ]; then
echo "Please specify movie filename"
echo "$0 [-n] "
echo "where -n is for 4:3 mode"
exit 1
fi
scale=368:208
while getopts n: o
do
case "$o" in
n) scale=320:240;;
[?]) print >&2 "Usage: $0 [-n] file ..."
exit 1;;
esac
done
let "x=$OPTIND-2"
if [ $x -gt 0 ]; then
shift $x
fi
src="$1"
title=`basename "$1"`
title=${title// /_}
title=${title%.*}
out=${src%.*}
out=${out}.MP4
echo Converting $src with title $title using scale $scale to output file $out
mencoder -ofps 30000/1001 -af lavcresample=24000 -vf harddup,scale=$scale -of lavf \
-oac lavc -ovc lavc -lavcopts aglobal=1:vglobal=1:vcodec=mpeg4:acodec=libfaac \
-lavfopts format=psp \
-info name="$title" "$src" -o "$out" 2> /dev/null
Do remember to chmox +x the file:
$ chmod +x convert_psp
Using it is pretty simple, just run the script and pass in the video filename:
./convert_psp <filename>
The script will generate a video file with an extension .MP4. Transfer this file to your PSP’s ‘VIDEO’ folder and you’re done!
One thing, I can’t and won’t claim all the credit for this script. It was not me who did all the work, and I remember copying the initial script from somewhere and I’ve retro fitted it for my needs.
If you’ve used Linux in the past, I am pretty sure that you’ve heard of the Ext3 file system. It is one of the most common file system format out there, used mainly on Linux based systems.
I’ve noticed something really annoying about how it handles large number of files in a single directory. Essentially, I have a directory with almost a million files and I found that creating a new file in this directory took ages (in the region of tens of seconds), which is not ideal at all for my purpose.
After some reading, and much research, I learnt that Ext3 stores directory indices in a flat table, and this causes much of the headache when a directory has many files in a directory. There are a couple of options.
One, restructure the directory so that it does not contain that many files. I did some tests, and in a default (untuned) Ext3 partition, each subsequent write degrades horribly past the 2000 file limit. So, keeping the items in a directory to within 2000 files should be fine.
Second, is to enable the dir_index option on the Ext3 file system. Run the following as root and you should find that it improves a lot. Do note that the indexing will take up much more space, but then hard disk space is not too expensive nowadays:
$ sudo tune2fs -O dir_index /dev/hda1
Finally, just use something like ReiserFS which stores directory contents in a balanced tree, which is pretty darn fast and you don’t have to muck around tweaking things.
If you’ve got your main partition as an Ext3, and can’t really afford to reformat it into ReiserFS, there might be an alternative: create a blank file and format that as a ReiserFS file system and mount it using loopback.
So, lets create the file first. This depends on how much data you need to handle, and in this example, I’ll just create a ~100MB file full of zeros:
$ dd if=/dev/zero of=reiser.img bs=1k count=100000
Next, format the file using ReiserFS as below. It will complain about the file ‘reiser.img’ not being a special block device (and we know that!). Just say yes and carry on.
$ mkreiserfs -f reiser.img
Finally, mount it where you would like to read/write files into it (need to do this as root):
$ sudo mount -t reiserfs -o loop reiser.img /tmp/listdir
You might need to do some chown so that your normal user can write into it. Moreover, if you need it to startup during boot, do remember to put it in /etc/fstab !
FYI, I used a Python script below to see how long it took to write new files:
import os
import time
count = 1000000
total = 0.0
for i in range(count):
if i % 1000 == 0:
print 'Creating %i' % i
start = time.time()
open('/tmp/listdir/%s' % i, 'w').close()
total += (time.time() - start)
print 'Avg is %0.8f' % (total / count)
I have an XBox (not the 360, but the really old one that is going for dirt cheap: £98.99 brand new from Amazon, and £29 from the Marketplace) that runs Linux (specifically Xebian) and it seems to work pretty nicely. Video playback is not the clearest or sharpest in the world, but it is sufficient, and I am content with the cost/quality ratio. I only use it to play videos and DVDs, and that’s about it. I don’t use it for surfing the web, email or anything more demanding. So, it makes sense to install a media center front-end like MythTV. The only thing is that MythTV is a resource hog, and only having 64MB of physical RAM it does not cope very well out of the box. However, there are some nice folks out there who have released a modified version of it that works pretty well. Now, considering that the prime (and only!) reason that I use my XBox is to watch videos, the natural component of MythTV that I would find useful is MythVideo. I have followed some instructions very closely, and always end up with the MythVideo module failing to load up due to some unresolved symbols. This is probably due to a library mismatch within one of the shared libs in MythTV. So, there I was with an XBox and I wrote a simple Java/Swing app that can invoke mplayer remotely (yes, yes, I know this is nasty!) on the XBox. Was doing this for about 1-2 years. The only problem with this approach is that I needed another PC/Laptop working to run the Java/Swing app. This was really inconvenient, and in one afternoon, wrote a simple GTK app. I call it MCE – Media Center E (I have no idea what ‘E’ stands for, just that it sounded natural when I was thinking of a name. Feel free to make something up!). The goal of the app is:
In a nutshell, it should be small and simple! So, I took an afternoon (2-3 hours) and wrote MCE. Now, it’s only in its works, but it is pretty functional as a first cut. To install it, you’ll need to install python, pygtk and any command line media player (like Xine, MPlayer, etc) If you’re using Xebian, or wish to run this on any other non-XBox machine running a Debian (Ubuntu, etc) based (or any other distro that uses dpkg and apt), all you have to run is:
$ apt-get install python python-gtk2
The latest release (version 1) is can be downloaded from: http://roopindersingh.com/releases/mce-1-release.zip There are some instructions in the readme file (under mce/1/docs). I’ll paste here for easy reading:
Prerequisite:-------------You will need the following installed: 1. Python (anything version 2.3 and above is good!)2. PyGtk (GTK2 bindings for Python)3. Mplayer (or any other command line media player) Installation:-------------1. To install, just unzip it somewhere:$ unzip mce-version-release.zip 2. Then go into the 'bin' directory:$ cd mce//bin 3. And add execute bits to mce.sh$ chmod +x mce.sh Configuration:-------------- All configuration is done in a file called 'mce.properties'.The file is always under the directory mce//config The parameters that you're interested in are:media.directories This is a comma seperated list of directories where your media files aremedia.extensions The file extensions that will be picked up by MCE. This is case insensitive media.player The command line player that will be playing the files. You will need to specify a single %s as that's where the media filename will be placed ingui.fs True - run MCE in full screen mode False - run MCE in windowed mode (if you're debugging it, etc) Running:--------To run it, go into the bin folder in mce//bin and run the'mce.sh' file:$ ./mce.sh
At the moment, you should be able to navigate the file system hierarchy using a mouse, keyboard or the XBox joystick and find your media files and play them. There is some gap around the list of files to allow some space for widescreen TVs. Screenshot:
Future releases will have:
And that is all I am going to make it do at this stage. I don’t want it to be one of these all singing and dancing media center frontends that can even make a cup of coffee for you, as that usually means bloat and resource hungry. License? Well, its got the source there – so, help yourself, I put it in the public domain. If you have any questions or requests about support, help, even features, do drop me a quick note by clicking here.
I managed to move emails and mail lists for Duzle.com and Midearth.co.uk to a faster and bigger server in about 3-4 hours.
My current setup, which I’ve had tweaked over the past couple of years, is pretty simple and straight forward:
All user settings are stored in a MySQL database, and this includes the SpamAssasssin settings and bayes learned tokens.
Postfix, Courier IMAP, Mailman was an easy migration, as most of the setup is in the MySQL database.
The most tedious part of the whole process was getting maildrop to work nicely with my setup above. FYI, a handy command for testing maildrop is:
maildrop -V 4 -d email_test@test.com < /etc/motd
For some reason, the Maildrop versions supplied in Ubuntu Gutsy are not backwards compatible to Ubuntu Edgy (at least I had to debug my maildroprc to get it working correctly).
After about 3-4 hours of hacking, copying, tar-ing, untar-ing and some careful planning to avoid missing out on emails during the migration, we’re back running on faster hardware and even more disk space!
Something I’ve always wanted to do – offer Subversion hosting, and for free. Why free you may ask? Well, in my opinion, comoditised things like Subversion (or any other version control) hosting should never, ever be charged for. Version control is something that everyone should use, and while the barriers to entry are getting lower (in the sense that the client tools are getting easier to use, and the concepts of version control getting easier to understand) every year, we still find that it is only being used by:
I probably would categorise myself at 60% on the first, and 40% on the second. Case in point – I really do put everything under version control – right from the most obvious things, like source code, to the not so obvious things like images, documents / reports and server config files. Ever since Subversion came out and offered binary file handling that actually works, I’ve been finding more and more interesting ways of using it.
Sadly, your average Joe will still be saving backups of their thesis on a thumb drive, and risk either deleting the thing, or potentially just loosing the thumb drive entirely. Or when they do end up making a mistake in the content (for e.g. rewriting a whole chapter), its just going to make them even more upset as they can’t roll it back to how it was before they rewrote it. Here is where I think version control is really useful and powerful – as long as Joe is in the habit of saving his work often. Tracking changes (comparing and reverting them) is such an easy task if the document was put under version control from the beginning.
Coming back to the hosting, I’ve offered this via XP-Dev.com, and it should be really easy to startup – just create a user account, login and start creating your repositories. Moreover it has virtually no space limits.
Give it a go, and as usual, if there’s anything you’d like to see, do just drop a note!
I’ve always found myself in the situation where I need to keep track of the list of things to do (or plan to do), especially when it comes to developing systems and managing projects. The problem is that there aren’t any tools out there that is simple enough to use, where the process of tracking does not become a significant overhead. I really liked XPlanner, but the problem it is that it’s trying to satisfy way too many people and the process of project planning/tracking can sometimes really be an overhead, especially when it comes to managing the stories and tasks.
The way I like to think of a project is that it is defined as a set of stories, and each story is defined as a set of tasks. To track a project, and ensure that you can meet your deadlines and delivery dates, the estimates (metric should always be in no. of hours!) should really be assigned at the task level – NOT at the story level. Stories should be best described as an atomic user case in a system, and the underlying tasks should define the implementation of the story. Think of it this way – a story is a plan/proposal and the tasks actually get you there.
So, ideally for me, a project planning tool should capture these fundamentals:
The are other nice things to know: for e.g. reports that tell you how far along you are in a project/story (and whether you’ll have to tuck your tail between your legs to senior management in the weekly review). But I’ll get back to this later.
XPlanner is the closes tool out there that manages to get most things right. However, it can be a little clunky when you actually get down to managing tasks and story. I used it about a year back and maybe it has moved on. The clunky interface is actually a burden on the developer. I was working on a pretty high profile project in my bank when I was using XPlanner, and I found that the developers were spending more time surfing through the tool, trying to look for one single feature (for e.g. adding hours to a task), rather than developing. Now, this is an investment bank, where 3 developers wasting 20 minutes each a day is a big major NO NO practice. Every minute counts! So, XPlanner was not ideal for me.
Coming back to project management. Let me reiterate the traditional constraints: scope, quality, time and cost. Very important to always keep this in mind when running any project. I have found that many project managers are actually missing these fundamentals when pushing for projects to meet deadlines and not reporting back to management/clients on these. There will be times where the clients are really impatient, and won’t really understand the constraints. But generally, most of them do understand it and will decide on which of the fundamentals they would like to have delivered (Funnily enough, I’ve found that at times quality is not one of the fundamentals in Investment Banking. However time is almost always one of that clients would like delivered).
From my experience in the past, you can at most keep 2 or even 3 of the fundamentals at any one time, and your tool that you use for managing projects should relay this information to you as clearly as possible. The problem here is that I have yet to find a tool out there that can do this seamlessly.
The other issue that I noticed is that the development team that I used to manage always worked on multiple projects. No matter what period of the year it was, there was always 2 to 3 concurrent projects running for a team of 5-6 developers. This is a constraint that I have seen in many project management tools out there. They tend to defined an iteration based on a project. So, project A will have a current iteration and you can add stories A1, A2 and A3 to it and kick off the iteration. This never worked for me, at work and on my own personal projects. I needed a tool that I could arbitrarily define an iteration as a delivery target for a number of projects and systems (time constraint). In each one of these iterations, we will deliver some functionality/fixes (scope constraint) given the current resource levels (cost constraint). So, what does this really mean ? Say, I had a release cycle of every 2 weeks. Within those 2 weeks I really need to convey to all the business lines that I can deliver parts of project A and B. Within those 2 weeks, I will get out functionally defined by stories A1, A2, B1, B2 and B3. I kick off the iteration and keep track on how they are progressing. There will be ongoing risks that I will convey back to the clients. My tool will tell me how much work is outstanding, and I can then give early warnings to the business that we need to readjust our constraints. Sound simple enough ? Again, no tool.
Long and short of it – I decided to write my own project tracking tool. I’ve put it up on XP-Dev.com. Its currently only single user: i.e. if you’re in a team, then you can’t really collaborate with each other on the projects. Its a little rusty here and there, but it does what it says on the tin. At any one point, I can see where I stand on all my outstanding work and projects.
I use it extensively for my personal projects and seems to work pretty good. The one thing I really liked about XPlanner (and I’ve shamelessly copied) is the task list. At any one point I can see the tasks that are outstanding to me from all projects and keep knocking them one by one off the list. This is follows the Getting Things Done time management (Next Actions list).
Give it a go, and if you have any requests, please do contact me and I’ll ensure that it gets done.
Subversion Hosting | Project Management | Bug Tracking | Free Subversion Hosting | Exentrique Solutions
Posts RSS Feed