It's important to establish your logo and brand as early as possible, and not to change it. Keep future modifications to your brand simple. The importance of this is to create an established presence that people will recognise in years to come.
Try to choose 2 colours (other than black and white) and stick to them. Work out the exact colour codes in RGB and CMYK and ensure that all printed and online works conform to those colours.
Choose a font, and stick to that font. Remember that the web has a limited selection of fonts.
When designing your website, brochures, business cards, letterheads and other stationery, make sure you conform to your chosen font and colours. Choosing 2 specific colours and using these for form backgrounds, highlights, headings, and links etc, will create a very professional looking and easy to read format.
When designing a logo, you can choose to have some one create a new logo for you, or you can save alot of time and cost by purchasing a "predesigned logo" relevant to your industry. Make sure they only sell the logo once. A good example of this is www.businessgraphics.co.nz