Posts Tagged ‘XP-Dev.com’

Freedom and All That

Tuesday, March 3rd, 2009

I’ve just finished posting a new blog entry on XP-Dev.com on the definition of Stories, Tasks and Bugs from XP-Dev.com’s point of view. However, the beauty of it all is that users make XP-Dev.com their own. They end up using it for various other purposes other than software development (and agile at that!). They are free to use it whichever way they see fit.

In my blog entry, I said:

However, this article (and future ones) are only guidelines – you are free to decide on how you’d like to use XP-Dev.com.

“Free to decide” – these words triggered a memory from the past.

Back in 2003 I was doing an summer internship/job with Lec here in the UK and was rushing back to their office from Victoria station in London. I was running late and did not have enough time to read the notice board on which platform I was meant to catch the train from.

I ran to the first person who looked like he worked at the station, and I asked him which is the next train to Bognor Regis (Lec HQ). Apparently there were 2 – one that was leaving at that moment, and another that was leaving in 30 minutes. I wasn’t too sure which one my ticket was for, and showed him my tickets and asked “I’ve got these tickets – which train can I board?”.

As soon as I finished my question, he immediately replied “Take which ever one you’d like – it’s a Free country”. He didn’t bother even looking at the tickets!

With adrenaline still rushing through my heart, I thanked him, ran to the train and boarded it. The train left a minute later.

I have always comeback to this little episode whenever I think of usages of a tool or even idea – let it be a handy tool (screwdriver, etc) or even an edit (one of the team asking whether its OK to use Vim instead of Emacs). I have a 2 second flashback to that moment in Victoria station and reply to them in the same manner – “Its a free country. Do whatever you want.”

And this is the beauty about building tools that people use, like XP-Dev.com. Everyone has their own way of using it and it should be just like that. No hard rules, everyone gets to do it their own way, and everyone is happy.

There’s a saying “With freedom comes responsibility” – so, just don’t do anything unlawful.

5 Tips for Building a Web SaaS

Sunday, February 15th, 2009

I have encountered tons of problems, some small, some large while building XP-Dev.com as a web software as a service (SaaS). To be honest, I was a little too naive and didn’t foresee some of these issues, and I really do hope it will help someone out there who’s thinking of building a SaaS for the masses.

1. Case Sensitivity

Case sensitivity of your unique keys is really important. For example, when you’re building a user database, you need to consider whether User1 is the same as uSeR1 and vice versa. Do note that email addresses are case in-sensitive and you’ll need to be able to cope with that in your application code. If you’re hosting your application on a Linux box, do remember that in general, Unix filesystems are case sensitive, and if you had a directory for User1 on your server, you could have another directory for uSeR1 as well.

On the database side, MySQL has a small bug nifty feature that will actually help you solve this headache a little. If you declare a column as a VARCHAR, searches on it are case insensitive, i.e. if you search for uSeR1, you will get back User1.

If you’re finding some weirdness around Hibernate, MySQL and case sensitivity, do have a look at by past blog about it.

2. Internationalization and Unicode

Just build everything on UTF-8, from the beginning, on each-and-every-file, on each-and-every-request. It will save you a whole load of headache later on when you’re considering releasing your SaaS to the non-english speaking world (and that’s a HUGE motha-** of a world that you don’t want to miss out on).

Use UTF-8 database tables. Depending on your installation, you’ll find that MySQL uses latin1, and that doesn’t bode too well with them accented and asian characters. The trick is to use the ‘CHARACTER SET‘ option when creating your database, and setting ‘charset=utf8‘ when creating tables.

Do use the awesome W3C HTML Validator to ensure that web browsers are reading your SaaS using the correct encoding:

3. Login/Register Lifecycle

Ahh yes, authentication! There’s a ton of research put into answering the question ‘How do I authenticate users on a website ?’. But my gripe is not about the authentication itself – it’s about doing the right thing after authentication.

Here’s a common scenario – User1 visits http://example.com/some/private/service/ which is an authenticated service – i.e. User1 needs to login to example.com to be able to access it. The problem is that some SaaS out there immediately redirect User1 to their ‘dashboard‘ or ‘homepage‘ on example.com – http://example.com/userhomepage.

This will frustrate users as they have to:

  • Remember the initial URL http://example.com/some/private/service/
  • Login
  • Retype http://example.com/some/private/service/ in the browser’s address bar
  • Press Enter.

The same applied if the user has not even registered for your SaaS.

The solution here seems pretty obvious – keep track of the last URL that a user hit before reaching your authentication pages, and upon successful registration or authentication, just redirect the user back to the original URL.

Most web frameworks will have support for this functionality in one form or another. Do look it up and get it in before the site goes live.

4. URLs and Permalinks

Keep everything in nice encapsulated URLs. This is a subjective area (that has been debated to an extent that it’s no longer funny), but I think having URLs that do not contain query paramaters are:

  • Easier to remember
  • Search engine/SEO friendly
  • Cleaner to regenerate in code

For example, instead of having:

http://example.com/some/private?service=login

You could instead have:

http://example.com/some/private/login

If you’re using a modern web server like Apache, Nginx or Lighttpd, they all provide some mechanism of rewriting URLs so you don’t have to modify your code too much.

5. Application Level Permissions Layers

Most SaaS are essentially database driven applications, and they all access the database under a single user. In some complicated setups, this can actually go to 2 users – one for reading and one for writing. An even more complicated setup, each user of the SaaS will have a database login.

All of these are essentially not enough.

And here’s why – in a world where a normalised database structure is all the hype, there’s a high degree of certainty that data for User1 sits on the same table as data for User2. As far as I know most databases don’t really have row level permissioning and hence, having to rely on your database as your permission layer just does not work.

There is one setup where I thought that it might work – give each user a new table or database. But clearly this is a solution that simply won’t scale.

So, what’s the alternative ? Embed it into your application code. The decorator or facade patterns are extremely powerful for implementing this. Moreover, you can do complex permissioning, for e.g. User1 can read the business object during weekends, but not when User2 is logged in at the same time. OK, fine – ts a bad example, but you get the point.

Why bother going through all this trouble ? Well, here’s a generic use case:

MyCalendar app is a web SaaS online calendar offering. Each user can have multiple calendars, and they are all private to the user. To retrieve a calendar, all a user has to do is visit http://example.com/calendar/<calendar id>/ where calendar id is an identifier on a database table.

Say User1 has calendars with calendar ids 240, 252 and 362. If MyCalendar app didn’t have application level permissioning, User2 would happily be able to view all 3 of User1’s calendar.

So, the natural question to ask is “Do users actually try to do that ?”. YES! They will. I’m not sure whether they are curious, or looking for a security hole, but you will find some users exploring the URLs. What I mean by that is, say User2 has a calendar id of 5442. He/She will try to visit the URLs for calendar id 5440-5449, even though there are no direct links to those calendars that they can see (except 5442).

Using a database driven web framework like Rails and Django is all well and good, but remember to implement some application level permissioning if you have any private data.

There You Have It

5 simple tips that will save you a ton of hassle if you’re building an SaaS. As always, feedback is appreciated.

Pricing Your Product

Friday, December 19th, 2008

My mind has been in a bit of a limbo the past few days, and it is to do with pricing software as a service (SaaS). I have blogged in the past that I do have a few ideas on generating revenue from XP-Dev.com, and pricing these ideas is an absolute pain. To top it all, I am really nervous about it. I have discussed it with a few people and I ended up getting more nervous than when I started the discussion.

However, Eric Sink has written down a few pointers about this. It does not give an answer to the golden question – How much do I charge for my SaaS ? but it does give some advice on how to tackle it. I think it is very well written.

Quoting Eric’s article:

You should consider all of these issues, and probably a few more that are specific to your situation. Look at the decision from every possible angle. Anything you read on the subject of pricing is merely an aid to your own judgment, not a substitute for it.

Amen to that! For now, I shall go back to thinking about pricing ideas for XP-Dev.com and keep my mind in a limbo.

Follow up on XP-Dev.com release

Thursday, November 20th, 2008

A quick follow up post on the recent release of XP-Dev.com. There were some bugs that needed to be resolved. Some pretty critical ones, while others were just minor annoyances. However, the dust should have settled now. I’m not saying that there aren’t any bugs left, but most have been fixed. Of course, if you do notice anything odd, please raise a support ticket and we’ll resolve it asap.

I decided to take it easy the past couple of days, as I’ve been working on XP-Dev.com like a maniac for the past 4 weeks. Just to give this some context – I began rewriting the whole of XP-Dev.com from 16th October 2008 and released it on 17th November 2008, which is about a months worth of work. I current site has 13k lines of Java (not including unit tests) and 2k lines of HTML/Velocity templates. There’s about 1k lines of XML, but most of that is just Spring configuration (NOT their web framework – I’m only using Spring for IOC), and I had a day job to take care of. There was plenty of refactoring done during this time, and these numbers are only the final figures. All in all – a crazy month, but well worth it!

It is a complete re-write with a whole new custom MVC framework and object relational mapper (ORM) layer. I’m seriously considering releasing the MVC framwork as open source (the ORM needs some work!), which I think is pretty darn good – writing new dynamic pages and forms is a breeze! Now, I’m sure a lot of you might be wondering why I embarked on such an ambitious journey ? Well, stay tuned as I will be posting a series of blog posts on how I re-engineered and re-architected XP-Dev.com, and the reasons behind those decisions.

The new XP-Dev.com has only been released for 3 days and I did get quite a few mails from users who expressed their gratitude. I can’t begin to thank all of you for those kind emails and support tickets – it is really good to know that there are plenty of you out there who are happy users, and that the past month has been well worth it! I’ve got many more ideas to put into XP-Dev.com, so, think of the recent release as the beginning of something new :)

More on XP-Dev.com

Wednesday, November 5th, 2008

Doug Bromley has made a comparison list of free Subversion Hosting providers on his blog, and XP-Dev.com is listed there (thanks Doug)! If you’re just skimming it, then it might look as if XP-Dev.com is a little “thin” on the features end of it. The odd thing I found was the “Tracking” portion had XP-Dev.com listed as a “NO”, which is not entirely true. Having said that, the project tracking portion is not as polished as I want it to be. There are more features in the works, and one of the main ones that most users have asked for is the Project tracking part to be multi-user.

I will keep everyone posted on the progress as soon as I can confirm dates, but it is really really close.

On a side note, I’m going to start a blog for XP-Dev.com at blog.xp-dev.com and wish to invite anyone out there who is enthusiastic about agile programming and software development (and delivery!) in general and can contribute some time to blog on best practices and other fun stuff, do drop me a note and we can discuss specifics!

Free Subversion Hosting

Saturday, November 1st, 2008

Many people over the past few months, have been asking the same questions over and over again about the services over at XP-Dev.com. I don’t mind answering them with the same answers, but I think it is time to put all of these questions into one place and discuss them.

Why are you offering Subversion Hosting for free ? Is it too good to be true ?

Let me set something straight:

I offer it free because I really do not believe that anyone should pay for something so simple to setup and run as Subversion.

Here is the reality: I setup Apache using mod_svn, mod_dav, mod_ssl and mod_auth_mysql once. Believe me: only once and never ever ever ever (ever!) touched it again. No, I am not kidding – only once! No tinkering needed, it just runs like Forrest Gump (no pun intended to all you Gump fans out there).

It does cost $$$ to host it, including my time to add more features to it. Disk space and bandwidth is getting cheaper. They are not free, but then again, if you average it across the number of users that I have on XP-Dev.com, the figure looks really, really small. It is a cost nonetheless, which I’ll try to cover below.

So, we’ve established it does cost money, how are you covering these costs ? Are you really rich ?

OK. I wish I was rich, but the truth is – I am not. I could claim I was rich and lie to you all, but then I would not get any glory every time I look at my monthly bank statements.

So, where does the money come from to pay for the services ? Well, at the moment, I am paying for it. But I won’t be doing this forever.

I have got a few models to generate revenue and these models will be implemented in the next few months. I can’t reveal them to the public just yet, but rest assured that the usage of Subversion and project tracking on XP-Dev.com will always remain free. This is how I started and envisaged XP-Dev.com, and that is how it will always be.

Free Subversion Hosting and Project Tracking on XP-Dev.com is a life-time guarantee.

You’re offering a free service. There’s a catch to it, right ? Are you selling our code to someone else ?

No. Nada. No catch. I am not a petty code trader. I don’t go around knocking on other peoples doors saying “PHP codez $4 per line! .. $3.50 per line! .. $3.40 per line! ..”. I could not even be the least bothered about what everyone else is coding. I have my own ideas to push forward and materialise (one of them is XP-Dev.com, there are a lot more in the pipeline).

So, your code is safe on our servers. No one else other than the ones you have permissioned are looking at your repositories. We do have backups that run every night and copied over off-site, but they are all encrypted before leaving the server.

I put all my code on XP-Dev.com. I am a consumer of my own service. I believe that anyone who offers a service should always be their own users/clients/customers. You should see your service from the customers point of view.

If someone else looked at my code and data, I’d be really worried. I respect that tremendously and try my very best to lock down the server.

What you see is what you get – WYSIWYG. There are no catches at all. Your code and data are safe. We have a “no prying eyes” and “mind your own business” policy.

OK. So it is a genuine service that is FREE with no strings attached. Then I suppose it will have to be an overloaded, slow service ?

Never! This is one of the things that come out from being a consumer of your own service. If the services do get slow, there’s going to be one really noisy, angry, verbal user – me. And I’m really scared of him.

On a serious note, I’d be disappointed with myself if the service ever comes to a unacceptable quality. At the moment it’s fast and quick and I intend on keeping it that way. If it every becomes slow, I’ll be there in front of the queue shouting.

I’m not too sure if this is a good thing, or a bad thing – I’ve only ever worked in the Front Office for Investment Banks building real-time (well, its near real-time) trading and pricing system. They are all high performance scalable systems. The systems I work on can cost a trader anywhere between $100,000 to $500,000 if latency went up a nudge above 10ms (yes, that’s milliseconds!). XP-Dev.com is a testament of my experience building & architecting these crazy systems (trust me, they are crazy!).  If performance degrades, it will be a major failure on my part and I’m a really proud person :) .

It is a great service. How can I help ?

This reply is a cliche. There are a few ways you can help.

If you are not a user, register now!

If you are a user, and have any problems, queries or just want to say thank you, then please tell me, or email admin@xp-dev.com. Every single non-spam email that goes there gets a reply. If you don’t get a reply in a few hours, then it’s probably SpamAssassin acting up. You should use this form instead.

If you are a user, or not even one just yet – you can help by telling your friends, mom, dad, brothers, sisters, relatives, neighbours, cats, dogs, fish and everyone else about XP-Dev.com. Digg it, Buzz it, Reddit. Do whatever. Just keep spreading the word. I really appreciate it.

If you have any other questions or concerns, please post them as comments to this blog entry, or do contact me directly.

Subversion Hosting

Wednesday, March 12th, 2008

Something I’ve always wanted to do – offer Subversion hosting, and for free. Why free you may ask? Well, in my opinion, comoditised things like Subversion (or any other version control) hosting should never, ever be charged for. Version control is something that everyone should use, and while the barriers to entry are getting lower (in the sense that the client tools are getting easier to use, and the concepts of version control getting easier to understand) every year, we still find that it is only being used by:

  • People who truly believe in it
  • Folks in the software industry – let it be developers, managers, or any member of your QA team (the examples carry on, but I’ll stop here)

I probably would categorise myself at 60% on the first, and 40% on the second. Case in point – I really do put everything under version control – right from the most obvious things, like source code, to the not so obvious things like images, documents / reports and server config files. Ever since Subversion came out and offered binary file handling that actually works, I’ve been finding more and more interesting ways of using it.

Sadly, your average Joe will still be saving backups of their thesis on a thumb drive, and risk either deleting the thing, or potentially just loosing the thumb drive entirely. Or when they do end up making a mistake in the content (for e.g. rewriting a whole chapter), its just going to make them even more upset as they can’t roll it back to how it was before they rewrote it. Here is where I think version control is really useful and powerful – as long as Joe is in the habit of saving his work often. Tracking changes (comparing and reverting them) is such an easy task if the document was put under version control from the beginning.

Coming back to the hosting, I’ve offered this via XP-Dev.com, and it should be really easy to startup – just create a user account, login and start creating your repositories. Moreover it has virtually no space limits.

Give it a go, and as usual, if there’s anything you’d like to see, do just drop a note!

Task and project management

Tuesday, February 26th, 2008

I’ve always found myself in the situation where I need to keep track of the list of things to do (or plan to do), especially when it comes to developing systems and managing projects. The problem is that there aren’t any tools out there that is simple enough to use, where the process of tracking does not become a significant overhead. I really liked XPlanner, but the problem it is that it’s trying to satisfy way too many people and the process of project planning/tracking can sometimes really be an overhead, especially when it comes to managing the stories and tasks.

The way I like to think of a project is that it is defined as a set of stories, and each story is defined as a set of tasks. To track a project, and ensure that you can meet your deadlines and delivery dates, the estimates (metric should always be in no. of hours!) should really be assigned at the task level – NOT at the story level. Stories should be best described as an atomic user case in a system, and the underlying tasks should define the implementation of the story. Think of it this way – a story is a plan/proposal and the tasks actually get you there.

So, ideally for me, a project planning tool should capture these fundamentals:

  • Each project has a set of stories
  • Each story has a set of tasks
  • Each task should have an estimate no. of hours
  • Each developer adds hours spent to a task
  • Each iteration should be defined as a set of stories (from any project)
  • Managing the above should be as simple as possible, and not an overhead to the actual development of the project

The are other nice things to know: for e.g. reports that tell you how far along you are in a project/story (and whether you’ll have to tuck your tail between your legs to senior management in the weekly review). But I’ll get back to this later.

XPlanner is the closes tool out there that manages to get most things right. However, it can be a little clunky when you actually get down to managing tasks and story. I used it about a year back and maybe it has moved on. The clunky interface is actually a burden on the developer. I was working on a pretty high profile project in my bank when I was using XPlanner, and I found that the developers were spending more time surfing through the tool, trying to look for one single feature (for e.g. adding hours to a task), rather than developing. Now, this is an investment bank, where 3 developers wasting 20 minutes each a day is a big major NO NO practice. Every minute counts! So, XPlanner was not ideal for me.

Coming back to project management. Let me reiterate the traditional constraints: scope, quality, time and cost. Very important to always keep this in mind when running any project. I have found that many project managers are actually missing these fundamentals when pushing for projects to meet deadlines and not reporting back to management/clients on these. There will be times where the clients are really impatient, and won’t really understand the constraints. But generally, most of them do understand it and will decide on which of the fundamentals they would like to have delivered (Funnily enough, I’ve found that at times quality is not one of the fundamentals in Investment Banking. However time is almost always one of that clients would like delivered).

From my experience in the past, you can at most keep 2 or even 3 of the fundamentals at any one time, and your tool that you use for managing projects should relay this information to you as clearly as possible. The problem here is that I have yet to find a tool out there that can do this seamlessly.

The other issue that I noticed is that the development team that I used to manage always worked on multiple projects. No matter what period of the year it was, there was always 2 to 3 concurrent projects running for a team of 5-6 developers. This is a constraint that I have seen in many project management tools out there. They tend to defined an iteration based on a project. So, project A will have a current iteration and you can add stories A1, A2 and A3 to it and kick off the iteration. This never worked for me, at work and on my own personal projects. I needed a tool that I could arbitrarily define an iteration as a delivery target for a number of projects and systems (time constraint). In each one of these iterations, we will deliver some functionality/fixes (scope constraint) given the current resource levels (cost constraint). So, what does this really mean ? Say, I had a release cycle of every 2 weeks. Within those 2 weeks I really need to convey to all the business lines that I can deliver parts of project A and B. Within those 2 weeks, I will get out functionally defined by stories A1, A2, B1, B2 and B3. I kick off the iteration and keep track on how they are progressing. There will be ongoing risks that I will convey back to the clients. My tool will tell me how much work is outstanding, and I can then give early warnings to the business that we need to readjust our constraints. Sound simple enough ? Again, no tool.

Long and short of it – I decided to write my own project tracking tool. I’ve put it up on XP-Dev.com. Its currently only single user: i.e. if you’re in a team, then you can’t really collaborate with each other on the projects. Its a little rusty here and there, but it does what it says on the tin. At any one point, I can see where I stand on all my outstanding work and projects.

I use it extensively for my personal projects and seems to work pretty good. The one thing I really liked about XPlanner (and I’ve shamelessly copied) is the task list. At any one point I can see the tasks that are outstanding to me from all projects and keep knocking them one by one off the list. This is follows the Getting Things Done time management (Next Actions list).

Give it a go, and if you have any requests, please do contact me and I’ll ensure that it gets done.