Form D is the document that startup firms need to file when they are raising money and are a type of firm that is exempt from registering with the SEC. While you can search those documents on the official SEC EDGAR site, this site is optimized just for finding a Form D. It has several advanced and filtering functions, including a "newest" filings feature. A range of startups are included, from high tech, social media, bio-tech, hedge funds, and those that are considered in "stealth" mode. Consider this site if you need to find information on all kinds of startup firms around the country.
When a company is private, but has plans to go public, they must file either a form S-1 or SB-2 with the SEC. The S-1's and SB-2's contain the full prospectus of a company at the time of filing, including financials, marketing, risk factors, competition, industry overview, executive compensation, and other information.
Depending on the state in which it is registered, a private company may have to file annual reports with the Secretary of State, although the actual reports may or may not be available on the web. Secretary of State offices may provide articles of incorporation, UCC info, etc. Use this site to access each Secretary of State's web site.