Archive for July, 2008

Amazon S3 is cheap, essentially limitless online storage. If this is the first time you’ve heard of S3, you’ll find everything you need to know about it online at the S3 homepage.

For those of you that are interested or already using S3, I have three questions.

1. How do you use S3?
2. What do you use it for?
3. What software do you use to interact with S3?

These questions come from my own experience working with S3.

The only solid client I’ve found for working with S3 is Transmit from Panic. Transmit allows me to work interact with S3 exactly as if it were an FTP server. It’s a beautiful thing.

On Windows however I haven’t been able to find anything reliable.

I’ve tried the S3Fox Firefox extension. It’s buggy as heck. I’ve contacted the author of the extensions author offering to help out with bug fixes and added features. I never heard back though. So that’s sad.

I’ve seen a number of other products, the seemingly most popular being JungleDisk. Almost all of these are geared for drive backups. That’s just not the way I need to use S3.

I’ve tried JetS3t. But it’s also pretty buggy. The kind of stuff it barfs on is not knowing how to map a folder on S3 to a folder on my local drive. That just totally baffles me.

WebDrive I have to say has made me pretty happy. Still it doesn’t have that “it just feels right” quality about it.

The main thing I use S3 for is storing my iTunes movies and music. I regularly work on 3-4 different computers. I don’t want to replicate my entire 50 gigs or so of iTunes data. So I just stash it up on S3 and then download songs as I want to listen to them.

We’re considering building an S3 client that better supports the type of more casual usage model that I described. Before we head down that road though we want to get some feedback from the rest of the world out there on how they use S3.

If you’d be interested in seeing new and better ways to interact with Amazon S3, let us know.

Where can I get the new version?

Head on over to the Wordpress Plugin directory and download the plugin.

Take a spin through the online documentation for installation and usage instructions.

What Changed?

This release adds proper rendering for headers with embedded markup.

Happy blogging.

Key features

In the business of developing software, we usually work with many people concurrently and interactively. This context has peculiarities which make some tools more appropriate than others.

Some of the key features in this context are:

1. Tasks are not static: Custom software development is more like driving on the highway than riding a train. Task definitions are more like the rails that keep you from going off the road than the rails that keep you exactly on track. As the software develops, clients get new ideas and requirements change.
2. Deadlines: Again, guard rails. Being able to see what tasks are most urgent is helpful when working on many things for many bosses.
3. Discussion is important: Each task begins as an intangible; then ideas and specifics develop and evolve. They are critiqued and reviewed. All mutations of the original idea should be tracked, archived, searchable and associated with related work items.
4. Tasks relate: It’s not at all uncommon to be asked to do follow-up work for a task that was completed months earlier. All of the previous information about the task is still relevant and provides an excellent refresher while moving into the new phase.  It is important to be able to relate new work items to the old/completed work items.
5. SCM (source code/control management) integration: All the code we write is tracked through a system that tells us what has changed in each version of a file. Files are “checked in” or “committed” when they’re working properly, or when we want to save some work. It’s tremendously helpful to be able to include a task identifier for committed work that can later be searched on to see what code changes relate to a work item or feature.
6. Status at a glance: From a management perspective this feature is critical. Who is working on what? Who is ahead of schedule, who’s behind? What deadlines are closest? Etc.
7. Clients: Multiple clients have multiple projects, each of which involves multiple tasks. We need to be able to provide access to all projects at all levels.
8. Notifications: In order to have running discussions related to tasks, it is necessary to be able to send/receive notifications when a comment is posted, a new task is created, etc. This needs to be configurable on a user-by user basis.
9. Team structure: When working in a team it is helpful to be able to define different roles and responsibilities which can be integrated into a workflow. “Lead Developer” and “Manager” would be two key roles. It is not necessary for every person to have a role. Only certain key responsibilities need to be assignable through some sort of role structure.
10. Triage: The task tracking system is a conduit for communication which allows you to see concretely the intangible element of progress on a project, one task at a time. The first step in measuring progress is defining the task. Certain team members are assigned the responsibility for “triaging” new tasks, making sure they are clearly defined and that expectations are properly calibrated on both the client and developer sides before work begins.
11. Invoicing: The ability to generate descriptive invoices from timesheets is essential.

Types of information

Some pieces of information are like signposts, staying rooted in place and guiding traffic around them. A wiki is the right tool for this type of information.

Some are like particles of magical, formless, floating pixie dust. A task tracker is the right tool to track this type of information. The key notion here is that progress can be quantified.

Some are the anonymous and non-descript traffic around the signposts. This is typically activity like people asking questions about how a feature works or asking for help solving a particular problem, configuring a system etc. This is “front-office” activity. It is questions asked of support staff as opposed to requests for features of or assigning work to developers. A forum is the right tool for tracking this type of information.

The lifecycle of the system goes like this:

1. A person asks a question in a forum.
2. Discussion ensues.
3. The discussion outlines a feature that should go into a subsequent release.
4. Tasks are created and related to the forum discussion. Features are described. Timelines are defined.
5. Work begins developing the feature.
6. Discussions related to the development process are tracked using the task tracker.
7. Releases are issued.
8. Just before a release becomes final, the completed feature is documented in the wiki.
9. The new feature is released.
10. People ask questions in the forum

Wash … rinse … repeat. I’ve left out the whole QA process for the sake of brevity. You can probably get a feel for how that might work for you, given the tools and types of information I’ve described.

Comparing the systems.

Now that the context is defined, how might various tools meet your needs?

In our experience, we have found it rare that a system satisfies all of the needs just described. Some systems are better for companies which develop a single software product (i.e. no support for multiple clients or invoicing). Some systems integrate the magic triad; forum, task tracker and wiki. Other systems don’t. Some systems license by the user, and are therefore expensive. Some systems are easy to install and use. Other systems require a lot of maintenance. There are many factors to consider.

Two main types of task trackers

Task trackers usually fall into one of two categories; “todo list” style or “urgency/priority/severity/percent-complete” (UPSPC) style, for lack of better names.

Todo-list trackers are loosely structured. They are geared toward small-scale and less-complex projects. Some people would argue that their simplicity makes them even more powerful than more structured alternatives. This is an arguable contention when viewed in general, but when discussed in the context of a specific task, the choice usually becomes clear. They are not the right choice for projects involving multiple contributors or management assignments.

UPSPC trackers usually enable developers to prioritize tasks. If this sort of system is used, it is critical that users understand the difference between “urgency,” “priority,” and “severity.” Even when they are understood, tasks are often mis-categorized.

Our Faves

Instead of exhaustively evaluating every possible system, we will give you what we consider to be the best of each type of tracker.

Jira (UPSPC style)

Product home page:

Jira

Made by:

Atlassian

Pros

• Well engineered features, such as a killer work log which allows you to easily track time as a developer, and to monitor and manage time spent by the rest of the team. Atlassian even offers a hosted integrated platform, called Jira Studio, which combines their products as a hosted service. This is a powerhouse package.
• Free licensing to open-source projects. This makes it a popular tool for open source java projects (particularly those centered around the products from Codehaus it seems).
• Available either to download and install or as a hosted service.
• Structured sensibly. It is integrated nicely with Confluence (their wiki product) and Fisheye (their source control visualization product). It has great features and macros, customization capabilities, and all sorts of other things.
• It is fantastic for a company building software products for sale.

Cons:

• Doesn’t well support tracking work for multiple clients.
• Doesn’t integrate well with invoicing systems like QuickBooks, Freshbooks, or Blinksale.
• Expensive. Commercial licensing is done on a user and feature basis. It is written as a J2EE application, and therefore should have a dedicated box for hosting.
• It is not great for a company building software for other people. As already noted, it doesn’t have the ability to easily integrate the needs of multiple clients.

Basecamp (a todo-list style tracker)

This task tracker is really the combination of two pieces of the Basebamp suite, “Time,” and “Todos.” If you are familiar with Basecamp, you will know what I am talking about. If you are not, check out Basecamp to see for yourself, and maybe even sign up for a free evaluation.

Product Homepage:

Jira

Made by:

37signals

Pros:

Basecamp is chock-full of features. The task tracking is almost an afterthought. The way we use Basecamp focuses on the forum-like “Messages” area. We discuss and refine ideas until a project or task scope is understood. We then move forward, communicating using these messages, and recording our time with Basecamp time log entries.

• It just works!
• Tons of features
• Well integrated with other applications. Many high-quality applications on the web have integrated with Basecamp, such as Beanstalk, Freshbooks, Harvest, and SpringLoops, among others. Thanks to Basecamp’s integration API, we have options in terms of how to use the various tools that make us more productive and reliable.
• Flexible pricing
• A total collaboration solution

Cons:

• Customizability is limited because it’s a hosted service.

Put another way, Basecamp makes us a better company.

The Rest

There are a number of other systems which we’ve collectively evaluated over the years, but haven’t worked extensively with.

These include:

• Trac
• Lighthouse
• FogBugz
• Bugzilla
• Intervals
• Mantis
• Perforce (via jobs)
• RT
• and many others …

Conclusion

A task tracker is a core tool for a development team. It is important to find (or build) a tool that can be shaped to exactly fit your project and working style.

Take some time to experiment. See which features help most. Give yourself some time to see how well the tool fits

We are always looking for tools that can improve our productivity and communication. If you’ve used a tool that we didn’t mention, please tell us, and the world, about it.

Happy tasking.

There’s very rarely one big thing which can make or break a software project. The all-night “hail mary” hack session that you or your developer thinks saved your project was, although heroic and well-intentioned, most likely not the major contributing factor to the success of your project.

Here’s the secret to successful software development projects.

Start with an idea. Discuss it with everyone.

Listen. Be tenacious but patient. Above all, be honest with yourself.

Stay flexible.

Reward your successes and learn from your failures.

Enjoy each small success.

What you will learn is that the reward from success is the team you’ve built, not the product.

Lead that team on to other projects. As you succeed with each one, larger projects will come to you.

Succeed with each one and you will realize that all successes are small because there’s always a larger one waiting for you down the road.

Nurture, challenge and care for that team and you will find yourself, a long time from now, reflecting on many small successes.

I posted a week or so ago about my solidifying realization that Java is no longer (if it ever was) one of the better languages for building web-apps.

The original article, Why Are There So Many Java Web Frameworks? can be found here.

One of the points made in that article was that as you mix different frameworks into an application you end up having to tweak or work around attributes of each framework that aren’t compatible with, or are undermined by features of another app.

For example …

A really good example of this, and the one that impacts me the most right now is probably how Spring and the hot-deploy capabilities of Tomcat interact.

When Tomcat reloads a class after the source file is changed, it dumps/clears the web context and reloads or rebuilds it. This is typical for the various application servers I’ve worked with (including Jetty, WebLogic and others). When Spring is being used, the context dump also dumps the Spring context, meaning that all of the beans that Spring manages need to be reloaded. Yuck!

Spring does have a refresh method which may avert some of the greediness of this process. I’ve never been able to implement it though because I don’t usually have control over the dispatcher servlet because I’m using Struts! This is yet another example of how frameworks compound application complexity when used together.

The net result for me is this. On an application with 3164 classes and 405 beans managed by Spring it takes me 63 seconds for the hot-deploy mechanism to pick up my changes.

So, in Java:

• I forgot a semicolon: Add a semi-colon, wait 63 seconds, refresh the screen in my browser.
• I need to change business logic: Change business logic, wait 63 seconds, refresh the screen in my browser.
• I need to tweak formatting: Tweak formatting, wait 63 seconds, refresh the screen in my browser.

Ugh!

In any interpreted language:

• Do anything. Inhale, refresh the screen in my browser, exhale, see the modified code in action.

The dream

Here’s what’s great about Java.

1. It runs everywhere.
2. There are an almost infinite number of libraries, tools (including IDEs) and frameworks available.

If you read almost any Java blog or talk to any junior to mid-level Java developer they’ll cite those two attributes as reasons why Java is so wonderful. There may be other reasons in addition. But you’re almost guaranteed to hear those two. Oh, you may also hear “It’s so much better than C++”. Who’s writes web apps in C++ though?

In my experience …

Unfortunately however, that’s not the whole story. If you start asking around, talking to Java developers with more experience, and particularly those who’ve worked in other language environments, particularly using interpreted languages, you’ll start to fill in more of the picture.

1. Application servers start to get more and more difficult to deal with during development as the size of the application increases. So, by the time I get to take advantage of all of the support for “enterprise-level” applications that Java promised, I’ve discovered more than enough difficulties to offset what I thought I would gain.
2. I’ve worked with dozens to hundreds of frameworks over the years. Every framework solves (or at least claims to solve) a problem. Unfortunately though, for every problem it solves, it creates 2 new ones. Sometimes I have to deal with them. Sometimes I don’t. When I do have to deal with them it’s usually twice as hard to solve the new problems as it would have been to just deal with the original problem that the framework solved.
3. I get bored and lose a lot of time and focus while waiting for my darned application server to redeploy 50 times a day. Fortunately though this leaves me a lot of time to check the news on CNN and see what’s new on YouTube.
4. Spring proxying is fantastic in terms of delivering a real platform for modular, pluggable, SOAs (Service Oriented Architectures). But it broke my debugger! My stack traces are full of references to proxies that I can’t pull up in my IDE. I have to know where the exception is occurring before I can figured out where to look to find the problem.

I could go on. But you get the point.

So have at it!

Live and let live I say. If you’re happy with your development life-cycle, the speed at which you develop applications and the tools available to make you more productive I’m truly happy for you. I would, however encourage you to look further afield. Try using some different languages. Try NOT using frameworks and just deal with the problems that the Java fundamentals (things like Servlets and JDBC as opposed to component-based UIs based on markup and ORM (object relational modeling) tools. You’ll be surprised how much faster your applications are. Whatever you think you’ll gain most likely won’t be worth the complexity and learning curve to attain it.

And lastly, if you’re a framework developer, keep it up! There may be a killer app out there that will boost Java past the yuckiness I’ve described. And you may be just the right guy to discover and/or develop it.

A final clarification

Just as a final clarification, lest you think that I’m a Java hater, I’m not saying that Java is bad. I’m just saying that it’s not the best tool, or even the best starting point for building applications for the web.

I feel that a lot of businesses have felt the need to embrace “enterprisey” platforms. At the same time, they’ve felt the need to embrace an increasingly web-based delivery model and Java has catered to that crowd. Businesses have embraced Java mostly because there weren’t any better alternatives, and or they didn’t have enough experience and knowledge about Java to be discerning or find alternatives. That is to say that there weren’t other platforms that catered to the crowd interested in “enterprise-scale web applications” to the degree that Java and Sun did. So Java was a safe choice for managers and developers starting out and looking for a technology to use as the basis for their career.

My thoughts in this article are intended to encourage IT managers and developers to look outside of the vast confines (isn’t that an oxy-moron?) of the Java landscape and consider options and alternatives for enterprise-scale computing platforms. It’s become simpler and simpler to achieve scalability (one of the biggest demands from the “enterprise” crowd) since Java became King of the Enterprise. And I think it’s time for businesses, particularly IT managers and senior/lead/principal developers and engineers to build on a platform of simpler, more nimble technologies to meet ever increasing demands for agility in the marketplace.