One of the top considerations of any web site owner should be speed.
In these days of broadband connections, people are used to near instantaneous results, and a slow loading front page is guaranteed to lose visitors.
There are several factors which control loading time.
One is of course server response times. There is little that can be done about that!
Then there is the core code of the site. WordPress is WordPress and there is little or nothing you can do to change it.
There are areas though that are well within the site owner’s control.
The one thing I would say about themes, from the speed perspective is that the more graphic the site, the slower it tends to be. A graphic rich theme is going to be slow, and it is worthwhile bearing this in mind when making a choice.
Plugins can be a site killer. I use a plugin on my sites – Plugin Performance Profiler (P3) – which is a good indicator of which plugins are holding the site back. Using that plugin I have managed to shave seconds off sites’ load times, by either indicating which plugins to remove or which to replace with a slicker version. In quite a few cases I have managed to find replacement plugins that have exactly the same functionality but are faster to load.
External links within plugins or extra coding (such as advertising or analysis0 are frequently the cause of sluggish sites, as the server has to wait to connect to those sites before continuing to serve up the rest of the main site. If your traffic analysis site is having an off day then your site will too.
As an example, I had a plugin on this site which displayed my blogroll on the sidebar along with their latest posts and date of posting. Before the page could load, the server had to contact each linked site in turn to retrieve information. I ditched the plugin and reverted to standard links and shaved 32 requests and several seconds off the load time.
These are probably the biggest contributors to “gross tonnage” of any site. Always reduce images in physical size and compression before uploading. A photograph that is say 3,500 by 2,000 pixels may look stunning but it will weigh many megabytes. Setting its display size may reduce the visible size but it will still weigh all those megabytes. If you only want it to appear at say 600 x 400, then reduce it before uploading. Also compress it as much as possible as many kilobytes can be shaved off an image that way.
Caching can have an enormous impact on page load times. It does so by generating static pages from the normal dynamic ones and therefore cuts way back on database requests, code requests and file loads. There are several excellent caching plugins which can be found on the WordPress site. While I haven’t tried all of them, the ones I have tried give facilities to allow code compression and browser caching which help greatly with load times.
A word of warning about caching plugins – I have noticed that Plugin Performance Profiler (see above) tends to give very poor responses for caching plugins. On closer examination though, the poor response times are almost exclusively within the Admin pages of the site. One would assume that not many outside visitors are going to reach in there?!