If the problem is vetting, really the best way is to get someone technical you trust (or with good reputation) to evaluate them. There are plenty of technical people whom you could hire for a day to give their opinion of any candidate.

However the real problem is usually finding the candidate and convincing them to want to join you. The people who you really want as a technical partner have a lot of other opportunities vying for their attention. You have to figure out how to stand out from the crowd.

It's hard to evaluate a technical candidate as a non-technical person. Probably the best thing you can do is find another technical person you trust to do the evaluation.

That said, there are some tertiary skills you can definitely verify. If somebody is going to be reporting to nontechnical personnel (which is probably true if you're trying to evaluate them, being a nontechnical person yourself) then things like clear communication, expectations management, and raising concerns before they become problems are all really important.

When evaluating a technical person who's going to report to non-technical management, I would ask things like this:

1. Show me a report you've written to advise upper management.

Somebody who has clear communication skills with management will have a track record where they've demonstrated those skills. Depending on confidentiality issues, they should be able to pull up an email, a document, a presentation, where they advised a nontechnical person. Maybe they were approached to evaluate a technical direction for a product, a team, the whole company, etc.

2. Tell me what you did when a project had serious problems.

All projects have problems sooner or later. The question is how you handle them. A good engineer should raise concerns, accurately, promptly, and clearly, and be ready to discuss possible changes in course. However some organizations shoot the messenger, so be careful not to judge a person for a whole organization's dysfunction.

3. Show me a project you're proud of.

A good engineer will have a demo of something to show you. Don't get caught up too much in how closely it matches what you're hiring them for--it's hard for a nontechnical person to judge whether software is alike or different under the hood. Instead focus on how they communicate. Do they explain the problem well? Do they explain the solution? Does it work well?

Keep in mind the candidate may have had factors outside of their control (customers, management, technology) that explain some deficiencies in what they've built. But how do they respond when challenged? Are they hostile? Are they empathetic? Do they start to problem-solve? Those behaviors are probably ones that will carry forward if they're hired.

It's hard to judge a programmer at programming if you're not one yourself. That's why you should go to another programmer you trust to get an opinion. But you can definitely judge his/her communication skills. And if your developer is reporting directly to you, that's going to be equally, if not more important.