Just because a plumbing contractor has not been in business for a long time does not mean that you should avoid them. Check references, get the plumber to give you a detailed bid and ask lots of questions. Especially for smaller jobs, you can usually be safe going with your overall impression from these steps.
When interviewing plumbing contractors prior to awarding your project, ask each contractor to give you a complete rundown of their safety, ethics and quality assurance practice. Also ask to see proof of insurance both for your property and for the people who will be working on your property.