Ramblings And Musings

A quick follow up post on the recent release of XP-Dev.com. There were some bugs that needed to be resolved. Some pretty critical ones, while others were just minor annoyances. However, the dust should have settled now. I’m not saying that there aren’t any bugs left, but most have been fixed. Of course, if you do notice anything odd, please raise a support ticket and we’ll resolve it asap.

I decided to take it easy the past couple of days, as I’ve been working on XP-Dev.com like a maniac for the past 4 weeks. Just to give this some context - I began rewriting the whole of XP-Dev.com from 16th October 2008 and released it on 17th November 2008, which is about a months worth of work. I current site has 13k lines of Java (not including unit tests) and 2k lines of HTML/Velocity templates. There’s about 1k lines of XML, but most of that is just Spring configuration (NOT their web framework - I’m only using Spring for IOC), and I had a day job to take care of. There was plenty of refactoring done during this time, and these numbers are only the final figures. All in all - a crazy month, but well worth it!

It is a complete re-write with a whole new custom MVC framework and object relational mapper (ORM) layer. I’m seriously considering releasing the MVC framwork as open source (the ORM needs some work!), which I think is pretty darn good - writing new dynamic pages and forms is a breeze! Now, I’m sure a lot of you might be wondering why I embarked on such an ambitious journey ? Well, stay tuned as I will be posting a series of blog posts on how I re-engineered and re-architected XP-Dev.com, and the reasons behind those decisions.

The new XP-Dev.com has only been released for 3 days and I did get quite a few mails from users who expressed their gratitude. I can’t begin to thank all of you for those kind emails and support tickets - it is really good to know that there are plenty of you out there who are happy users, and that the past month has been well worth it! I’ve got many more ideas to put into XP-Dev.com, so, think of the recent release as the beginning of something new

Folks, there’s a new version of XP-Dev.com out. This release features a new platform (which I will blog about soon!) and some key features that everyone has been asking for profusely, namely:

• Subversion imports and exports (tons, upon tons of users have asked for this)
• Allowing anonymous (public repositories) checkouts
• Multiuser project and task management (tons of users have asked for this)

There are some obvious bug which I will sort out in the next few days. However, under the new platform, extending and adding more features to XP-Dev.com will be a breeze (and unit tested of course)! There is a whole lineup of features coming up, and will keep everyone posted about it. These are exciting times for XP-Dev.com and we really appreciate all the support that’s been given to us.

I personally would love to see XP-Dev.com being the best agile tool out there, and we’ll get there!

If you’re a current user, give it a whirl - any feedback will be great - good and bad!

If you’re a new user - register now and see what it can do to improve your development deliveries.

Anyone else notice how 80% of your traffic comes from 20% of your referers ?

To be honest, for me and the sites I run (including XP-Dev.com) the rule is more 90%-10%. The top 10% of the referers bring in 90% of the visitors.

I saw this on digg.com a while back and had a good laugh. Thought I’d share it here (original site).

Seth Godin’s latest blog has some real wise words:

Awesome news that Barack Obama is the new US President! He has an uphill battle ahead of him, and I really hope he’ll be able to turn things around, for the folks across the pond and the rest of the world in general.

Doug Bromley has made a comparison list of free Subversion Hosting providers on his blog, and XP-Dev.com is listed there (thanks Doug)! If you’re just skimming it, then it might look as if XP-Dev.com is a little “thin” on the features end of it. The odd thing I found was the “Tracking” portion had XP-Dev.com listed as a “NO”, which is not entirely true. Having said that, the project tracking portion is not as polished as I want it to be. There are more features in the works, and one of the main ones that most users have asked for is the Project tracking part to be multi-user.

I will keep everyone posted on the progress as soon as I can confirm dates, but it is really really close.

On a side note, I’m going to start a blog for XP-Dev.com at blog.xp-dev.com and wish to invite anyone out there who is enthusiastic about agile programming and software development (and delivery!) in general and can contribute some time to blog on best practices and other fun stuff, do drop me a note and we can discuss specifics!

I have blogged in the past on a very sensitive issue - Eclipse vs IntelliJ. I come from the IntelliJ camp where I strongly feel that IntelliJ really improves developer efficiency tremendously. It might be a little opinionated, but here’s the drill: I have been working at my current job (no, its not a career, but that is content for another blog) for roughly 8 months, and I can only use Eclipse, for a multitude of reasons. So, I might be able to claim that I do use Eclipse and IntelliJ a lot.

Here’s what I found when comparing Eclipse to IntelliJ: Eclipse has way too much clutter. It does not refactor right. It is really slow in auto-completion. It just does not offer the right suggestions during auto-completion. The list goes on. It could be that I’m facing these issues because I’m using Eclipse 3.3. I really should upgrade to Eclipse 3.4 and hopefully some of these might go away.

My IDE has to be really fast, and has to be smart enough to give me the right auto-completion. My IDE has to be able to help me develop real quick. Eclipse falls short - way short. I tend to develop much faster (about 2x productivity) thanks to IntelliJ’s pretty smart auto-completion, and somehow the code comes out cleaner, as I don’t have to worry about the IDE screwing up big refactors. I end up spending less time typing, and more time refactoring.

However, the one thing that I always liked about Eclipse is that it supported various other languages and IntelliJ fell short in the category. Well, not for long - IntelliJ is catching up and now with version 8 - there’s a ton of features added with new language support. This list is just the tip of the iceberg on the new features.

Grab your evaluation copy, and try it out. Tips to using IntelliJ properly:

• Shortcuts, shortcuts, shortcuts! Learn all the keyboard shortcuts. Use the reference card (Windows + Linux, Mac) and memorise it.
• Stop touching that mouse! Every single time you move one hand from the keyboard to the mouse you have just wasted a few seconds of good coding time. Break this habit.
• Learn to type fast, without looking at your keyboard.
• Get in the habit of refactoring. There maybe once, or twice in my 3 years of using IntelliJ that it failed on my when refactoring, and I use that bloody rename shortcut like it is going out of fasion. The more often you refactor, the cleaner your code will become. It will be more maintainable as well.
• When you’re editing something that is not exactly code, like XML for instance, don’t use notepad or some old crummy text editor! Chuck it into IntelliJ and you might get a suprise that it does auto-completion for you.

Some useful sites to help you get your head around IntelliJ:

• IntelliJ product page
• IntelliJ 8 product page
• IntelliJ blog
• IntelliJ Early Access Program (EAP), with the IntelliJ 8 EAP.
• Jetbrains home page

Many people over the past few months, have been asking the same questions over and over again about the services over at XP-Dev.com. I don’t mind answering them with the same answers, but I think it is time to put all of these questions into one place and discuss them.

Why are you offering Subversion Hosting for free ? Is it too good to be true ?

Let me set something straight:

I offer it free because I really do not believe that anyone should pay for something so simple to setup and run as Subversion.

Here is the reality: I setup Apache using mod_svn, mod_dav, mod_ssl and mod_auth_mysql once. Believe me: only once and never ever ever ever (ever!) touched it again. No, I am not kidding - only once! No tinkering needed, it just runs like Forrest Gump (no pun intended to all you Gump fans out there).

It does cost $$$ to host it, including my time to add more features to it. Disk space and bandwidth is getting cheaper. They are not free, but then again, if you average it across the number of users that I have on XP-Dev.com, the figure looks really, really small. It is a cost nonetheless, which I’ll try to cover below.

So, we’ve established it does cost money, how are you covering these costs ? Are you really rich ?

OK. I wish I was rich, but the truth is - I am not. I could claim I was rich and lie to you all, but then I would not get any glory every time I look at my monthly bank statements.

So, where does the money come from to pay for the services ? Well, at the moment, I am paying for it. But I won’t be doing this forever.

I have got a few models to generate revenue and these models will be implemented in the next few months. I can’t reveal them to the public just yet, but rest assured that the usage of Subversion and project tracking on XP-Dev.com will always remain free. This is how I started and envisaged XP-Dev.com, and that is how it will always be.

Free Subversion Hosting and Project Tracking on XP-Dev.com is a life-time guarantee.

You’re offering a free service. There’s a catch to it, right ? Are you selling our code to someone else ?

No. Nada. No catch. I am not a petty code trader. I don’t go around knocking on other peoples doors saying “PHP codez $4 per line! .. $3.50 per line! .. $3.40 per line! ..”. I could not even be the least bothered about what everyone else is coding. I have my own ideas to push forward and materialise (one of them is XP-Dev.com, there are a lot more in the pipeline).

So, your code is safe on our servers. No one else other than the ones you have permissioned are looking at your repositories. We do have backups that run every night and copied over off-site, but they are all encrypted before leaving the server.

I put all my code on XP-Dev.com. I am a consumer of my own service. I believe that anyone who offers a service should always be their own users/clients/customers. You should see your service from the customers point of view.

If someone else looked at my code and data, I’d be really worried. I respect that tremendously and try my very best to lock down the server.

What you see is what you get - WYSIWYG. There are no catches at all. Your code and data are safe. We have a “no prying eyes” and “mind your own business” policy.

OK. So it is a genuine service that is FREE with no strings attached. Then I suppose it will have to be an overloaded, slow service ?

Never! This is one of the things that come out from being a consumer of your own service. If the services do get slow, there’s going to be one really noisy, angry, verbal user - me. And I’m really scared of him.

On a serious note, I’d be disappointed with myself if the service ever comes to a unacceptable quality. At the moment it’s fast and quick and I intend on keeping it that way. If it every becomes slow, I’ll be there in front of the queue shouting.

I’m not too sure if this is a good thing, or a bad thing - I’ve only ever worked in the Front Office for Investment Banks building real-time (well, its near real-time) trading and pricing system. They are all high performance scalable systems. The systems I work on can cost a trader anywhere between $100,000 to $500,000 if latency went up a nudge above 10ms (yes, that’s milliseconds!). XP-Dev.com is a testament of my experience building & architecting these crazy systems (trust me, they are crazy!).  If performance degrades, it will be a major failure on my part and I’m a really proud person .

It is a great service. How can I help ?

This reply is a cliche. There are a few ways you can help.

If you are not a user, register now!

If you are a user, and have any problems, queries or just want to say thank you, then please tell me, or email admin@xp-dev.com. Every single non-spam email that goes there gets a reply. If you don’t get a reply in a few hours, then it’s probably SpamAssassin acting up. You should use this form instead.

If you are a user, or not even one just yet - you can help by telling your friends, mom, dad, brothers, sisters, relatives, neighbours, cats, dogs, fish and everyone else about XP-Dev.com. Digg it, Buzz it, Reddit. Do whatever. Just keep spreading the word. I really appreciate it.

If you have any other questions or concerns, please post them as comments to this blog entry, or do contact me directly.

Since going live with XP-Dev.com a few months ago, there has been a nasty little bug that I’ve been trying to chase down where a weird exception is thrown in some obscure situations. When I say “weird”, I mean “I look at the code and keep saying to myself: this exception can’t possibly be thrown here”. Now, the exception is very domain specific and its almost pointless for me to attempt describing it here, as that will take a few posts, rather than just the one.

In a nutshell, a user could login to XP-Dev.com using valid username and password credentials, but then found that he/she could not perform any tasks. It turns out that this only happens if a user logs in using a slightly different case for his/her username. For e.g. When user ‘developer‘ logs in using ‘Developer‘, he’ll basically hit this bug.

The problem is two fold:

First, MySQL stores varchar columns as case insensitive. So, when you run a SQL query like:

select * from Users where username='Developer'

MySQL will return the rows with username ‘developer’.

This is not a problem per-se, as I’ve found this design feature pretty useful when attempting to do web oriented work in the past. The web in general is case insensitive, and system facing the web should reflect that. There is no difference between XP-Dev.com and xp-dev.com.

So, from this perspective, its not really a security bug as well, as you can’t really register the user ‘Developer’ anyway and attemp to ’steal’ the other user.

Here comes the annoying second part of this fiasco.

I use Hibernate as a Object Relationship Mapper/Persistence Layer on the current version of XP-Dev.com. Carrying on with the MySQL example above, pulling out a User object from the Users table using Hibernate can be done as follows:

User user = (User) org.hibernate.Session.get(User.class,
                                             "Developer")

The problem is that the User object returned will have the member ‘username’ set to ‘Developer’, rather than ‘developer’ as it is on the database. The object that Hibernate returns to you does not reflect the database state at all.

Coming back to the bug: Whenever a user logs in, I keep his username in memory attached to his session on the server-side, and refetch a fresh new User object on every HTTP request. The problem was that performing any tasks failed because it relied on strong Foreign Keys on the database tables. For e.g. if user ‘developer’ had 3 repositories, and logs in using ‘Developer’, I will refetch ‘Developer’ using Hibernate and get the username ‘Developer’ back, rather than ‘developer’. The user ‘Developer’ does not have any repositories and the permissions layer that I wrote (which is case sensitive) will throw a nasty exception when user ‘Developer’ tries to access repositories for the user ‘developer’. The permissions layer should be case sensitive as it’s a very critical part of the architecture and should be as restrictive as possible.

It’s nasty the way Hibernate does not return the exactly values back on the data object as it is on the database. The work-around is to use something like below:

org.hibernate.Session.createQuery("from User where username = ?")
                             .setParameter(0, "Developer")
                             .list()

Magically, Hibernate DOES return a User object with username set to ‘developer’. This approach has an extra overhead of creating a list on every call, but I suppose I’ll have to live with that for now.

On the new version of XP-Dev.com, I got rid of Hibernate and Wicket. I’ve had growing pains with Wicket as well, but that’s a rant for another blog post. Hibernate is just too bloated, and I can’t afford the performance overhead.