This is probably one of the biggest expenses of owning any online or brick and mortar store. However, there are a few ways you can advertise inexpensively.
While some website owners have great luck with pay per click, I reaped very little in using this form of advertising - no matter how big of a budget I set. Affiliate programs can be very lucrative, as long as you are are careful of the types of ads you allow on your website. And while SEO companies promise a lot, there is only so much they can do for you - and it can be VERY costly.
Here are some thoughts on inexpensive ways to advertise within your own community:
Send flyers to the local schools. Check with the administration first - sometimes they will send home advertisements with the students if you include a discount coupon. I know that the schools around me will place flyers on the table in the teacher's lounge. Do this around the holiday season!
Offer to help the schools out with a fund raiser!
Contact the Chamber of Commerce. They allow guest speakers on different subjects - make yourself know to the community.
Offering items for newborns? Contact some of the local OB/GYN offices. Some send sample products, etc., home with their new mothers. Add something small and inexpensive that advertises your products!
Having your website optimized is actually one of the best things you can do - make sure that all of the elements of your site are set up correctly - check out the WAHM websites offering these services. I am a huge proponent of hiring others working from their homes - they have the same skills as some of the larger companies offering the same services - for a much more reasonable fee, since they have less overhead. Check references and look at their client's websites to see how well things are "working".