Karl asked me to compile all of the FAIL Meters (Tom Calloway) and come up with one final review. Here are the results with details elaborating each factor they accrued FAIL points for:
Eucalyptus received a total of 10 points of fail due to size and compression. This suggests a great amount of obsolete code. The first step would be to dedicate an individual to removing this obsolete code and standardizing the software.
Source Control: +5pts
Many people said the documentation for the source control was extensive, hard to follow, and dependent on a programming background. It is slightly more acceptable to require a background in programming when it comes to source control, but not to the point where even those with a background struggle to get through it. Remember, the goal is to make this project as open-source as possible. We want even those with no background able to contribute. If the Wiki is so long that a new user is instantly turned away by the sheer size of the source control documentation, it probably needs to be edited.
Building From Source: +15pts
A few people said that it could be difficult to understand the documentation if you had little programming background. So I tagged on 5 more points of fail. This is probably the easiest part to fix. Within a short time the documentation could be improved and use language that essentially anyone could understand. This would also allow new users the ability to utilize Eucalyptus without requiring them to have a background in programming.
I also tagged on 10 points for containing hand-written scripts. This isn’t a world ending add-on. In fact, we might want to waive this, considering the type of project Eucalyptus is. It’s a specialized architecture project, so you can expect some custom shell scripts to be included.
System Installs: +10pts
Many people tried to get Eucalyptus out of the 10 pts, explaining that it ‘recommends’ installing in ‘/opt’, but you can choose a different directory. At the end of the day, it doesn’t matter. It installs into a directory that it doesn’t/shouldn’t have too.
I added a small amount of fail to releases because of the time frame of releases. There isn’t any established consistent release date for new versions. Releases come periodically, every 7-9 months or so. Establishing a deadline and project goals would greatly increase Eucalyptus interest and drive. People tend to procrastinate without deadlines. Having a deadline with project goals gives the community something to look forward too, and the engineers something to work on.
Eucalyptus started as a University project, and wasn’t open-source until some time after. I gave them a reduced +5pts because they have earned it in regards to how open-sourced they are now.
Final: 50 Points of Fail “Babies cry when your code is downloaded”
I don’t think this is a bad place to be. You can see a clear direction for Eucalyptus, and there is a lot of potential. However, there are a lot of areas that can be improved.
The FAIL Meter assessment allowed me to take a look at the entire Eucalyptus project with a completely unbiased opinion. Interestingly, the fail meter doesn’t dive deep into the source control. Eucalyptus could use some improvements in the source code, particularly documentation and standardization.
The bigger problem lies within the management of the project. Eucalyptus doesn’t follow any release deadlines. Without release dates, software engineers don’t have a vested interest in the project. They can get around to their assignment when they feel like it. The community doesn’t have anything to look forward to and be excited about. If you think about a successful open source project like Ubuntu, not only do they have planned release dates, but their releases are reflected in their version numbers. There is always a lot of hype when a release is around the corner.