share
Stack OverflowShould "inability to code under pressure" be a valid excuse when writing code in an interview?
[+40] [38] Jim McKeeth
[2009-06-29 19:11:04]
[ interview-questions ]
[ http://stackoverflow.com/questions/1059948/should-inability-to-code-under-pressure-be-a-valid-excuse-when-writing-code-in ] [DELETED]

I've come up with what I believe are realistic problems to work on during an interview. Frequently I have candidates respond that they cannot code under the pressure of me watching them code (via Live Meeting or Locally). Is this a valid excuse for inability to complete the task (or taking too long) during the interview? If so, what can I do to decrease the pressure during the interview process?

It would seem that being unable to program under this kind of pressure could be problematic in typical employment because there are times when we as developers are fixing code when our manager is standing beside us, or during internal demos with product management. Additionally there is also the pressure that is typical with programming jobs that comes with deadlines (yes, we all hate them) and bug fixes.

Edit: I do my best to not "breathe down their necks" but I don't exactly abandon them during the process. Maybe I will take the "get the hell out of there" approach.

(44) It's a different pressure when you're trying to get employment than when you're trying to meet a deadline. - Nosredna
(11) even if you don't physically leave the room, have something else to do/keep yourself busy, so it doesn't seem like you are just waiting for them. ie if you'd rather make yourself available for discussion/questions, that should be fine... - Nader Shirazie
(1) @nader: I find the someone in the room to be even more pressuring, because you know that the other person is watching what you do, and is probably killing time waiting for you. At least if he's elsewhere, he might be doing email or work. - Uri
@Uri: to be honest, I agree with you, its easier if there's no one there. I guess I was being more speculative -- if the interviewer really wants to remain in the room, is there a way they can do it, and still keep the interviewer comfortable. You're probably right that in almost all cases, the answer is no - Nader Shirazie
(2) I'm not sure "get the hell out of there" is a good idea. Unless you give a trivially simple problems, there's a big chance the interviewee will need some hints along the way. They might get stuck in the middle. So, if in 20 min you come back and the problem is solved, it's good. But if it's not, what does it tell about interviewee? Not much. They made a mistake, that's all. You can't reject someone just because they made a mistake. By contrast, if you actively participate in code writing (give hints, explain requirements if needed), you can learn much more about them. - Igor Krivokon
I wouldn't "get the hell out of there". Instead, you might think about what it is you are asking them to code. Maybe simplify things a bit to keep it a little more focused. - Chris Lively
Just to re-emphasize Nosredna's point. Let's face it, getting job or should I say, trying to get a job is usually a miserable experience. After all the mind games, running around, and playing salesman; It's one thing to finish up a product, and another trying to get into one. Maybe I'm a bit too sympathetic, but its rough as it is - Rev316
[+60] [2009-06-29 19:16:55] Uri

I personally think that it should be (though my definition of pressure is that there is somebody watching), though I'm biased - I have the same problem.

I am an experienced programmer, and I have also done my share of programming under pressure (e.g., in academic environments such as solving a final exam as an undergrad or teaching as a grad) and believe I did it fairly well. However, I find that when I am writing code while somebody is watching my every tiny move, it is extremely distracting and I fail even the trivial stuff.

In my case, I try to figure out what the interviewer is thinking, and I keep looking for little tells. This is fine for Hold'em, where you have two cards to look at, but not so fine for programming.

Also, I personally tend to initially write with crappy syntax (on a whiteboard) and then fix little things, rather than do a clear line-by-line. Again, in an interview, that's a problem because I am worried that as soon as I don't close that brace or add the semicolon, I will look bad.

In addition, I often find "think aloud" to be distracting. This is acceptable in pair programming, but not in individual programming.

Furthermore, I have been in interviews where while I am writing, the two interviewers keep chatting between them on how they solve the same problem. Imagine that you are trying to code while two other people are reading each other's code aloud to the same problem. This is among the "scare the interviewee techniques".

Finally, some people don't do well with paper or with whiteboard. I am one of them. I like knowing that I can insert lines easily and type at the speed that I think. When I am confronted by a whiteboard or paper, it takes me ages to handwrite, and in my case I also can't really read my handwriting so everything is trickier. Give me a computer and I'll do fine.

So in a nutshell, my view is this: Wanting to make sure your interviewee perform under pressure is a very good idea. Time limits are also a very good idea. However, your prospective hire will never have to code with someone looking over his shoulder like that.

Therefore, give him the task, tell him you'll be back in 20 minutes, and then get the hell out of there!!!. Leave them alone. Give them a phone number for questions or for letting you know when they're done.

Note that I do agree with Drew that it's important to isolate the person from the aid of Google and code samples. I don't consider that pressure, though.


(21) +1 for get the hell out of there - Nader Shirazie
Exactly, I said the same thing in my answer, basically. It's not about how well you react under the pressure of someone watching you. It's about the product you deliver. They are two very, very different things. - William Brendel
+1 for get out. - mquander
How does "academic environment" == "programming under pressure"? - anon
@Neil, I think Uri was referring to the pressure of programming with a room-full of students watching. - MatW
(3) @Neil: How quickly you accomplish a programming task directly relates to how much time you have to refine / double check / test your work which in turn relates to your chances of getting an acceptable grade which again influences whether or not you loose your academic scholarship (and possibly your sanity) which finally determines how much more loans you need to take out or how much more cash you and your family need to scrape together and in fact whether you will even finish the degree. I'm mildly offended that some people feel that school does not induce pressure in the student. - sweeney
I suggest it induces slightly less pressure than a trading floor full of screaming traders wondering why their exchange connection has gone tits up. - anon
(2) This is what we did, pretty much. We put the canidate in the room with a test and left him alone for four hours. The test was a rough thing to finish in the time allowed (I did it barely and I designed the problem) but it seems to be a pretty good way to check skills. We haven't gotten anyone who can't code at all since then, anyway. - quillbreaker
@quillbreaker: I am not a very big fan of the four-hour-test on the onsite interview thing. An onsite is designed to get to know the person. Keeping them in a room for four hours for hours and letting them fail before they even met anyone means that you didn't have enough early screening and just wasted a plane ticket. - Uri
@Neil: Unless you are a quant, there are few jobs that require you to "be on the floor" like that. At the worst case, your client calls every 30 minutes. He doesn't sit and read every line of code. - Uri
@Neil: Academic environment means you have a final test, and you get 10 minutes to write a solution to a problem. That is not that different from an interview, especially when the questions tend to fall in the same "reverse a linked list" group. But in academia, it's you and the paper. - Uri
@Uri I've worked on trading floors - have you? - anon
@Uri I've also worked for two of the UK's biggest Universities (and attended two others) so I think I understand the academic world. - anon
@Neil: I'm well aware of your record, but aside from our mutual animosity, I'm not sure why my definition of deadlines in finals not acceptable for you. As for trading floors - no, I didn't, but I argue that most jobs are not like that, and OP didn't indicate that he was interviewing people for such a job. - Uri
@Uri My problem with you is that you come out with stuff like "Unless you are a quant.." when you abviously know nothing about the business area imn question. Stop doing that, stop pretending that being an academic is "hard" (anyone that has worked in academia knows it is very, very easy) and I will have fewer problems with your answers. - anon
@Neil: Look, I know very well that something I said in the past (not even sure what) set you off and since then you constantly follow my activity on this site in an attempt to prove that I'm a pretentious idiot. Now I'm sorry I've given you that impression, and I don't expect everyone on SO to like me. I can go crying to mommy (or to Jeff Atwood) or I can choose to not give a damn; I choose the latter. If you avoid nitpicking every word that I said, and read it just like everyone else here, then you will see I just meant that most jobs are not like that, and that academic=exams. Fine? - Uri
@Neil: And by the way, I work for company in the area of trading tools so I do have some idea about the field. Fine, it's not only quants, there are other people who are on the trading floor. Still, I would argue that for 99% of devs, pressure is very short deadlines or urgent bugs, not actual trading floors. Fine? Now just start ignoring me and find somebody else to try and intimidate all the time. I will not leave SO just because you don't like my posts. Add a UserVoice request to block certain users. Thank you. - Uri
@Neil: And BTW2, you are the only person on the top SO users list who has given more downvotes than upvotes to people. In comparison, other top users like Jon Skeet, Steven Lowe, etc., have about 10 times more upvotes than downvotes. So maybe you are a little bit more critical than average. - Uri
@Uri If you think I am stalking you in some way, you are mistaken - this is the first post of yours I've commented on in weeks. And if you can't bear being criticised, I think the Internet in general. and sites like SO in partiular are not for you. - anon
@Uri Regarding the upvote/downvote thing: I assume the quality of postys here follows a normal distibution. Therefore I should downvote as many posts as I upvote. Which is what I do +/- a couple of points. And I have no interest in being "average". - anon
@Neil: I haven't posted much in a long time (see my rep graph) since I started a full time job. However, if I look at most of the negative comments I received over the past years, they are from you. Hence, you might not be stalking me, but are either following me more closely than other SO users, or are the only one who is this outspoken about my stupidity. Either way, we are both too old for this silliness, and someone with your credentials should just live with the fact that there are dumb people around. I am not afraid of criticism, or I wouldn't be on a site with downvotes. - Uri
@Uri If the only negative comments you are getting are from me, then you are indeed living a charmed life - I get multiple downvotes every day ("not suprisingly", you would no doubt say). However, I am not going to stop downvoting or commenting adversely on your posts, when I find such actions merited, simply because you don't like it - that way lies censorship. Feel free to complain to whoever you like about this. - anon
@Neil: Obviously, I am getting my share of negative downvotes as well, but nobody else bothers to comment so much, and nobody else bothers to nitpick every word or try to take it literally or fight a loophole in anything. I don't always feel this is contributing much, though I openly admit it when you make a legitimate point. I also upvote enough of your answers when I agree with them, just as I hope that you do with mine. Neither of us is going away, so let's keep it that way. But you have often said you feel I am in the wrong profession, you're entitled to that, but keep it to yourself. - Uri
@Uri As you obviously want the last word.... - anon
I'm ok with four hour tests as long as I can bill for it - Billworth Vandory
1
[+35] [2009-06-29 19:26:43] womp

Whenever someone asks about working under pressure, I always think of this:

alt text

[Taken from the Dilbert archives]

Working under pressure is hardly a measure of the quality or expertise of someone. Bad programmers can work under pressure too. Forcing someone to work under pressure like that might be the equivalent of making someone speak in public, who knows?


2
[+30] [2009-06-29 19:47:26] eeeeaaii [ACCEPTED]

I have had a lot of experience giving interviews and creating coding tests for interviewees. A lot depends on how you stage the test. If you slam them with a hard question right off the bat, even smart and experienced people will have a brain fart and do badly. You don't want geniuses smacking their heads when they leave the building saying "oh yeah, duh! dumb mistake" and then going on to work somewhere else. Give them a super-simple, easy question first to let them warm up to the situation and get used to the fact that you're breathing down their neck while they code. I mean REALLY easy, like from an "intro to programming" course. Once they get some confidence answering an easy question, give them a harder one -- keep upping the ante until they crack. And of course, if they can't answer even the easy question, that tells you something about them as well (that they lied on their resume, for example...).

Other tips:

  • don't distract them too much while they are coding, but make a joke once in a while to make them feel more comfortable.
  • don't fuss about little syntax problems. Even good coders miss stuff that the compiler would catch.
  • don't require them to know the names of API functions or other crap they can always look up if they want (e.g. what are the arguments and types of the fooBar function?). IMO this is a particularly silly thing to ask because it tells you nothing about the candidate except maybe what their job performance would be if the internet went down and all API references, help files, were to simultaneously be deleted from their machine. If I'm interviewing somewhere and someone asks me this it would just make me want to work somewhere else.
  • (instead) give them problems that give you insight into their problem solving skills and knowledge of basic algorithms and concepts. You know, computer science stuff. You don't want code monkeys, you want engineers.
  • give them fun problems too, because when you have a good candidate, the interview is also an enticement to come work there (in case they have other offers)

p.s. I forgot the most important thing. For crying out loud, take the time to give them a computer and notepad++ or the equivalent. Don't make them write it out with a pen, that's just cruel.


(1) +1 Good advice for avoiding some of the pitfalls of interview coding questions. - Keith Smith
+1 for Notepad++, this is why I hate CS exams! - ChickSentMeHighE
3
[+26] [2009-06-29 19:17:47] Nader Shirazie

Yes, coding under pressure shouldn't be a problem, but interview pressure is a very different animal to deadline pressure. So I wouldn't be too quick to put them both together...


Exactly. Very different kinds of pressures...someone may code well under normal pressure, but not with all of the (significant) interview pressure. - Beska
4
[+15] [2009-06-29 19:15:08] drewh

Absolutely not. You want to see what a potential candidate can come up with without the aid/crutch of Google in front of them while coding. It seems like you've found a pretty good filtering mechanism as a matter of fact--assuming your coding problem isn't too off the wall, the fact that a candidate refuses to/can't perform under these conditions is a definite "NO HIRE" red flag.

To lower the pressure, perhaps just put them in a conference room with a laptop and an internet connection (if access to the net won't spoil the problem) and tell them to work out the problem as best they can in a set amount of time. This avoids the whole "over the shoulder" pressure some people may feel.


(24) Why is using Google such a bad thing? There is blatantly stealing code and then there is finding similar solutions, modifying, and implementing them. If I can save an hour by taking 20 minutes to research it, why not? The solution is faster & likely close to as good. Chances are somebody has done something similar before. - bdwakefield
(char limit) Honestly, I don't care if you are the most amazing programmer in the world. If you are slow it is bad for business, projects, schedules, deadlines. Now, if the solutions are total crap (and should be easy to spot) then that is a problem. Especially if the programmer doesn't/can't fix it or doesn't know where to start. - bdwakefield
My point is that too many times during phone interviews, candidates simply attempt to google the answer to a question instead of thinking about it. If the problem isn't easily google-able, then by all means, make it and any other tool that will help available. - drewh
(4) bdwakefield: Because interviewers are interested in distinguishing how good a prospect is at figuring things out, and how good they are at finding things out. - mquander
(2) The third thing that you should provide in the cconference room is the interviewers phone extension number - it is higly likely the interviewers spec will be faulty and need clarification & discussion. - anon
(1) If your interview questions can be answered with merely a quick Google search, perhaps you're asking the wrong questions? Do you want to know if they have memorized sizeof(short) * CHAR_BIT >= 16, or do you want to know if they can solve real problems? - Roger Pate
(6) "Here mister Carpenter... build me this shelving unit out of this 4x8 sheet of plywood as here as a test to your proficiency... and no you cannot use a saw" If Googling candidate is a definitive no Hire for you then you are not worth my time as a company - Newtopian
Solving the problem is more important than (or should be) having tons of things memorized. There is a very distinct difference between something that is difficult and something that is useful. Look up the Nerd TV (why no new episodes?!!?!?!) interview with Max Levchin (co-founder of paypal). - bdwakefield
Why would anybody be asking them to code something over the phone? That doesn't seem to make sense to me. I guess if it works for your hiring process but it seems kind of silly. I would much rather be able to see the answer after sitting them in a room and leaving for a half hour. Use the phone to see if you want to give them the chance to come in and interview for real. Should be more personality / goal based and maybe a quick overview of their skill set? - bdwakefield
I've been known to forget the most trivial things, like the arguments to main for example. Should having to look it up real quick disqualify me from the job? I'm also terrible at math. Any programming question you ask me that involves math is going to have me googling for the formula. If math isn't a key part of the job, does it matter I have to google for the area of a circle? - wadesworld
5
[+11] [2009-06-29 19:22:19] William Brendel

A deadline is one thing. Having someone breathing down your neck is another.

  • Deadlines are useful in the sense that they set a goal for completion.

  • Breathing down someone's neck is only useful if you want to irritate them and make them panic.

I would make sure they know the basic syntax of the language using some trivial problem. Then I would give them a significantly more complex problem and tell them to whiteboard a solution, without writing any code.

If they come up with a good solution on the whiteboard, tell them to take a laptop, go to a quiet empty office, implement the solution, and show it off in one hour. That is one way of giving some a deadline, without breathing down their neck.

If your company pays coders to design application while jumping out of airplanes, breath down their neck. I would not want to work for a company like that though.


(2) Oh no! I would rather eat my own head than go into a Starbucks! - anon
I was actually waiting for someone to say that. I almost wrote "go to a coffee shop", but I wanted to see if someone would write something about Starbucks :-) - William Brendel
(2) Well, at least I've made someone's day! - anon
(1) Say, bye-bye laptop!! - jmucchiello
@William: I agree minus the starbucks. Just put them in a conference room with a laptop and no network. - Uri
Why not a quite part of the company, eg company canteen? - Darknight
@anon : I agree, but I must ask, which head? =)))))))) - Andrei Rinea
6
[+10] [2009-06-29 19:14:45] Matt Bridges

Are you literally over their shoulder as they code? That's a little off-putting imo. It's one thing to give someone an hour and a task and then review the code afterward. Quite another to hover over them while they declare each variable.

If you're talking about just whiteboarding an algorithm with pseudocode, that's a perfectly reasonable interview expectation.


7
[+10] [2009-06-29 19:45:55] Ramin

I'm personally of the opinion that pop quizzes and programming questions have no place in the interview process. The interview is the chance we have for seeing an individual eye-to-eye and figuring out if they fit into the group or the company culture. It's a waste of time giving them kiddie quizzes under artificial circumstances. I like to find out more about the person, what makes them tick, what they like to work on, what was the most creative way they solved a problem, how interested they are in learning new stuff, how they communicate, stuff like that.

And if a manager hovers behind a programmer's seat while they're trying to think, that manager deserves a swift kick in the ass. Last thing you want is a programmer making an error because you're hovering back there making them nervous.

If you really want to find out how they code, have them print out a few pages of code they're most proud of. Or hire them under a no-fault 3 month probation. If by the end of that time it's proven that they can't produce code get rid of them.

But fer crying out loud, don't waste time with stupid dog tricks.

My $0.02 (and I speak from experience on both sides of that fence).


8
[+4] [2009-06-29 19:39:43] Jeremy Frey

You want a developer who can code under the pressure of x, where x can be any one of: tight deadlines, ever-fluctuating / fuzzy requirements, angry customers, broken code, insert the headache-of-the-week-here. But rarely is x = being observed.

Furthermore, think Heisenberg: you're going to change the outcome simply by observing.

You can have a developer who's fantastic working with ever-changing requirements, working with angry customers, working with extremely tight deadlines, and you want a person like this. But that developer might also have performance anxiety.

If performance anxiety is a dealbreaker for you, then you should watch your interviewees code. Otherwise, you should find ways to simulate the pressures you do care about, and let them solve your problems unobserved.


9
[+2] [2009-06-29 19:15:24] jjnguy

If a job were a completely stress-free or pressure free environment, then I'd say it would be a good excuse.

However, that isn't the case. Good programmers can program under pressure.


(3) Good programmers work for good projekt-planners and so do not have to code under pressure. - BeowulfOF
(7) Bad programmers can program under pressure too. Programming under pressure is hardly a measurement of quality. - womp
10
[+2] [2009-06-29 19:27:18] Don Branson

If they're going to be in an environment dominated by pair programming, they better be able to code with someone watching over their shoulder. Even if not, if I had two equally qualified candidates, I'd rather have the one that can code under pressure.


even with pair programming, the expectation of your peer is to help, not only evaluate. Getting the interviewee to roleplay one person out of a pair in a pair programming exercise would be cool -- but it has to be peer style interaction... - Nader Shirazie
fair enough. when I worked in an environment where pair programming was enforced, there was quite a variety of pair dynamics, including some outside the bounds of what you describe - that is, the peer would often act as judge rather than helper. presumably, jim is not a terminal sphincter like my boss was, or he wouldn't even care to ask... - Don Branson
11
[+2] [2009-06-29 19:45:19] DaveE

Early in our interview process, we give the candidates an abstracted real problem from our application & give them a couple (two, maybe three) days to return it. We do want to see what they can do but have found that the during-the-interview situation it unrealistic. They have to return that & have it evaluated before we proceed.

If I'm on the interview team, they'll have to whiteboard a solution to a different problem, but no code. This is to evaluate their thinking, not their ability to output compilable code under pressure.

And besides, they're going to be able to look things up if they come work for us anyway...


12
[+2] [2009-06-29 20:19:56] JonnyD

The mindset that allows us to produce excellent code is very different from the mindset of of an interviewee.

We can all kick ass when we're in "The Zone", but the mental concentration and focus required to get there won't be found during a nervous interview.


13
[+2] [2009-06-30 03:11:35] Tony Heupel

So much is context dependent. For the exact reasons you discuss, I tend to focus in this priority order during interviews:
1) Have them speak their thought process out-loud as much as possible and evaluate their problem-solving approaches in general--right, wrong or otherwise.

2) As they go through their solution and thought process, how do they interact with me--ESPECIALLY WHEN THEY GET STUCK!

3) The merits of their particular solution, if they reach one.

I make the solutions VERY acheivable, but when people get stuck, I'm more concerned about their ability to try repeatedly try new approaches and then to work WITH ME as I prod them forward and ask them questions and throw out ideas.

We all get flustered sometimes and don't show our best abilities EVERY interview, so it there is a small amount of validity to it. However, some questions are very basic syntax/framework/OOAD type questions that they should just nail. In the other "interesting" questions, I evaluate primarily the interactions during the problem-solving process and the solutions a close second. This, to me, is a better gauge of my ability to work with them in the longer term, especially in crunch-time pressure situations in the "real world."


Well said. I love the idea of giving them an interesting/engaging question rather then some ...Could you do it in log(n) crap... - Rev316
14
[+1] [2009-06-29 19:16:28] bdwakefield

I would say that it depends really. How long are these interviews? How complicated are the problems? How realistic is it that you would expect the programmer to be able to come up with a solution in 30 minutes or less without having any idea what might be asked of them.

Sure in my current application I can have answers about what is going on (if not actual fixes) in less than an hour, sometimes 30 minutes. I am familiar with the code and it's behavior for the most part. I have been working with it for over a year now but 85% of the code was not written by me.

I think it might be unrealistic depending on the question asked, not in all cases, but at least for some.

How many "green" programmers are put in situations where they are likely to have to answer directly to "the boss"? I have always seen them paired with a more experienced programmer for a period to kind of help them and guide them.


15
[+1] [2009-06-29 19:25:16] Samuel Carrijo

I like the "leave him and come back later approach", but I also think you'll be missing much this way. I've been through some interviews like that, and they also gave me time to review my code, and point out what I think was wrong.

I think it all depends on what you want to see. If you want just to know how good the "end code" of a interviewee looks like, than the "leave him and come back" should be good enough. If the way he approaches the problem is also a big issue, then you should stay watching, or maybe use a screen recording approach (less pressure this way).

Another suggestion, is starting with some really easy problems, that should be solved in 5 min or less, and help him get more comfortable with this idea. Then you start adding some (but not much complexity)


I find it tends to be 'leave him and wait for him to come and find you after three minutes', but unfortunately most my recent interviews that have bothered with written technical questions haven't left me alone. - Pete Kirkham
16
[+1] [2009-06-29 19:27:49] Jeremy McGee

Much depends on what the candidate will be expected to do if they're hired.

If they're going to be pair programming, or working in an environment where colleagues frequently peer review by working together, then this kind of pressure is perfectly justified and (as others have said) this is a good indicator of whether they should be a "no hire".

If they're going to sit in a room all day and program by themselves, then this isn't really a fair judge of what they shall do.


The pressure of performing in front of an interviewer isn't the same as the pressure of coding with colleagues once you have the job. People settle into jobs quickly and that pressure disappears after a short while. I don't really think it's a great comparison. - Andy E
Peer review doesn't involve someone silently looking over your shoulder and judging you on the outcome. - Pete Kirkham
From a candidate's point of view, if my interviewer doesn't engage with me while I'm coding during the interview -- that's a NO JOIN! - Jeremy McGee
17
[+1] [2009-06-29 19:28:49] Kevin Horgan

I think being able to handle pressure is important, things go wrong in production systems and programmers sometimes need to think fast and clearly to solve or at least contain a problem.

However, I think the pressure of being in an interview is artificial and I would not rule someone out because they could not code a solution with me sitting across from them. If they can do it then great, if not I would look for some sort of compensation control such as walkthrough some of your code and having them explain how it might be refactored or whatever just to confirm they are literate in the language / frameworks you need.


18
[+1] [2009-06-29 19:35:02] Makach

Remember, when you're talking to a candidate it's a mutual interview, you talk to your candidates and they talk to you. You can quiz them but if you like but don't ask them to work for you. I don't believe it's the place for such things.

If you want them to code, use the probation period, give them a simple task and see how they perform.

We had some success just handing out some code to look at (real simple stuff) where we discussed pros/cons and what was going on in the code. You quickly realize which candidate you want to give a probation period.


great point about quizing. - IAdapter
19
[+1] [2009-06-29 19:51:15] Imagist

Inability to work under pressure doesn't make a person a bad programmer. But an excellent programmer who doesn't know Java is an inappropriate hire for a Java job. Likewise, an excellent programmer who can't work under pressure is an inappropriate hire for a job where the programmer needs to work under pressure. Given the description of the job, I think that the ability to work under pressure is necessary.


20
[+1] [2009-06-29 20:13:44] User

I haven't really done code writing in interviews, but many other funny things. And I can tell you this: it's not the pressure of being watched or getting the job, it is the perception of an ingenuine environment or in other words a play-situation. It is especially bad if you have psychos (ops, I meant psychologists) sitting there. I personally was never able to behave naturally knowing it was just a play.

The candidate's declared inability to write code under observation can be due to the critics he'is be getting from you on the very first line of code. If you are not criticizing but helping to create a friendly relaxed environment and it still does not work, then it might be a bad sign.

Anyway, only a test period will tell you whether a candidate has the qualities you are looking for.


There is one more thing to it. Work under pressure vs. work in loud environment. I am personally able to speed up my activities when there is little time or something has to be done urgently, but I am incapable of concentrating in a loud environment with talks, calls etc. I suppose these are two different qualities and you should consider them separately.


Not sure who dug this down, but.. Well said! - Rev316
21
[+1] [2009-06-29 20:16:00] Jarrod268

You could also quite possibly scare off a good developer. "Is everything I do going to be this scrutinized? Will I need a hall pass for the bathroom? What happens when my kids are sick and I need to work from home?" At that point its not about pressure but the idea of "Big Brother" always being there.


22
[+1] [2009-12-01 04:48:16] Scott Ellsworth

I have given over 100 interviews. Some have gone well, some not-so-well, and of the failed interviews, a few have gone poorly for reasons that have nothing to do with the candidate's ability to do the job. If I detect such a problem, I put in my writeup that I do not have confidence in the screen, and my score is discounted. If I do not detect it, but the committee sees that I reported a completely different experience than the other people who screened, then they dig deeper, often choosing to discount a score completely out of left field.

Part of my job as an interviewer is to correct for the whiteboard effect. Almost nobody codes well on one, but we have a very good baseline about the kinds of mistakes a whiteboard causes. Those that impact the job are important, but those resulting from someone not being able to hit control-space should just be ignored. After a while, every interviewer I know gets a feel for which mistakes matter.

You will find the odd interviewer that cares about syntax over substance, but again, the committees have read a great many summaries, and they know each interviewer's foibles. They comment back to us on interviewer failures. Among other things, I have had hiring committees tell me that they found a question not sufficiently useful as asked and answered, or that a question is now on major interview question web sites. In addition, we read each other's questions, and how they scored the result, and we do talk.

It is true that the one hour job interview does not match the experience of working for us. I do not think I have had to actually code a sorting algorithm in the last five years, but I have had to make judgements about a library's quality, and one way that I do that is to look at the sorting algorithms they chose, along with a host of other indicators of how much the implementors care, and how well they implemented. How a candidate approaches a problem tells me a great deal about whether they could make those judgements. How they implement that solution tells me even more.

Hint: the best answer, if given what is essentially a sorting question, is "I use the library implementation. In Java, the standard Collections.sort impl uses the (whatever) method for ordering to implement a (whatever) sort that behaves this way for fully sorted data, thusly for mostly-sorted data, and such-and-so for unsorted data." Then be ready to implement something. This gives me maximal data to make my call.

To be honest, the whiteboard also forces candidates to focus. The times we have let people bring in their laptop and favorite IDE, it has actually not gone that well. They tended to get lost in the details, and to not find a 'good enough' solution in the time they have.

Scott


Does the hiring committee then report to the steering committee too? Sounds like a fun place to work. Sorry for the strawman. - Billworth Vandory
23
[0] [2009-06-29 19:28:44] stevedbrown

It's really the unforturnate reality of interviewing - not all good people can do well in an interview. I do think that great people will be so good at coding that they will be able to write some code even if they got two minutes of sleep the night before though.

That's just an interview... it's not a great way to decide on people, but if you don't have working experience with people, it's the only way you can do it.


24
[0] [2009-06-29 19:31:58] JP Alioto

In this case, I try and put them at ease, tell them they have as much time as they need, take deep breath and go forward. The reason I don't leave the room is that I want to see more than the end result. For example, if I leave the room, they could go on to SO, copy a Jon Skeet answer change some variables and present it as their own. Also, I am trying to evaluate what the skills of the person are. So, I want to see what they get stuck on, how they resolve the issues they come upon as well as how they interact with the IDE.


25
[0] [2009-06-29 19:37:53] Marco van de Voort

I don't think there is a golden way. If you really have the feeling the person is acting, show him the door. If (s)he really is nervous, just take your time, and see if (s)he improves if the immediate pressure is away.

At one point you have to ask yourself what you want. Quality employees, or quick interviews.


26
[0] [2009-06-29 19:43:11] Paul Chernoch

As a candidate, I prefer the whiteboard approach, where the interviewer allows me five or ten minutes to get started, then begins to ask what is my approach to the problem, why I used certain classes, asks if I overlooked a certain part of the problem statement, etc. and prods me in the right direction. This is a good test of:

  1. Problem solving & analytic ability
  2. Language syntax & memory
  3. Software design (& trade-offs)
  4. Handling criticism
  5. Work under Pressure

The interviewer often changes the problem in mid course and asks how the candidate how they would adapt their program to handle the new requirements, what kinds of cases the algorithm would perform poorly on, etc.

But on the flip side, if you are looking for an integration coder, ability to find other people's solutions, internet examples, etc. in a hurry and work that into the project is important. The whiteboard approach is good at testing algorithmic ability, but not research acumen. Perhaps a different kind of interview task (like offering a laptop with a wireless connection) would be better to test that kind of ability.


27
[0] [2009-06-29 20:28:33] filmjbrandon

By this test I'd ask, are you penalizing a programmer because they want to eliminate distractions?

Personally, I think that coding is actually more art than science (at least it is to me), and problem solving skills are not the same as engineering talent.

Evaluating a coder for whether they can solve a problem quickly doesn't necessarily mean that they've been able to think things through well enough to solve it properly, or with an eye towards future problems, or even elegantly.

The crutch of Google and maybe stackoverflow ;) might be fair to eliminate, but I agree with an earlier answer that the best thing to do is leave them alone. I know I personally am most productive when I can focus on the problem, the solution and the technique without much distraction.

That doesn't always/ever happen naturally, but if you're trying to test a programmer in an ideal condition so you can evaluate their skill level, I'd say that leaving them alone is better. Ive yet to work in a team, where despite the pressures of management, the engineering team wasn't still able to put on headphones and tune out the world once they started writing code. In some cases, they wouldn't respond to IRC or IMs either until they solved their problem.


Good answer. My only issue would be that the real world is rarely an ideal environment. If you're able to continually work in one, congrats! - Mat Nadrofsky
28
[0] [2009-06-29 20:32:26] Mike Dunlavey

I have a problem with interviewing techniques like this.

  • I think an important skill in programming is how not to be hasty, but to think clearly, consider alternatives, and not engage in guesswork.

  • Another problem is that often people are asked about arcane language-specific "hoohaw", like when to make a "friend" class or the the working of virtual constructors, rather than how to analyze a problem.

Fortunately, for all the jobs I've had, my reputation was good enough - I haven't had to go through HR.


29
[0] [2009-06-29 20:37:49] Lextori

I think it's entirely reasonable to ask a candidate to demonstrate their jobs skills. The one thing I would be wary of, is make sure that you are actually testing their job skills and not just being a pain, or adding unnecessary pressure.


30
[0] [2009-06-29 23:18:20] byanity

I perform technical interviews on a regular basis and have some coding samples prepared ahead of time. Honestly it's a mixed bag of responses. Some people plow through them in a minute and others freeze up under pressure.

Interviewing for a job is stressful. Some people can handle it and others cannot. Depending on the position I'd factor this in, but usually for a straight coding situation if they freeze up under pressure I'll just ask some additional code-specific questions to ensure they have the fundamentals necessary for the job.


31
[0] [2009-06-30 00:48:54] Anthony

I have done some of my best coding under pressure. So yes, sit accross the table and glare at them and see what happens. Of course I am also the world's worst debugger under pressure. I have solved so many bugs driving away from a job I wonder if I should bill for the time.

Eric Lippert had a blog about his interview style [1] where he presents a problem (with many different ways to solve) and see's how the person goes about solving it. This would give a you a good idea of what they know, how they would tackle a problem, and cut through any manure someone attempts to serve

[1] http://blogs.msdn.com/ericlippert/archive/2009/03/20/it-s-not-magic.aspx

Thanks for the shout-out. Here are some more articles on similar topics: blogs.msdn.com/ericlippert/archive/tags/Interviewing/… - Eric Lippert
32
[0] [2009-06-30 04:06:07] Keith Smith

There are times when you need to know if a candidate actually has basic coding skills. How are you going to find out?

There are always going to be times when the interview is the only opportunity to figure out if the candidate can actually write code. I wouldn't consider "inability to code at an interview" a valid excuse for a candidate to avoid this. I'm not going to play ogre, and deliberately try to make things hard for them. I'll encourage them and tell them that I'm not going to fuss about syntactic details. I'll talk them through it, or leave the room, if I think it will help them. But at the end of the day, we need to hire people who can get the job done. If a candidate can't show us that they can do the job, we're not going to hire them.

Having said that, I'm certainly interested in other ways to get at this information. Sometimes there are obvious ways to learn more about a candidate, such as references. But that only works if the reference is somebody you know and trust.

I don't like having them bring sample code, since that's a problem for folks whose current (or former) employers own the code. And the fact that they have code doesn't tell you anything about how it was written. How long did it take to write? to debug? What kind of review and input did they get from others?

I like the idea of hiring somebody for a "no fault" probationary period. I'd be curious to know how this has worked for folks in practice. I can't imagine the typical HR department signing off on it. Also you have to be sure that you and your team will be ruthless enough to let them go if things don't work out. It's easy to say that, "We'll get rid of them if they turn out to be a bozo." But in my experience, the real problems aren't the bozos, but rather the second-tier programmers. The nice but mediocre programmer probably won't raise any red flags during their probationary period, but then spend the next few years delivering late, producing more than their share of bugs, and generally getting in the way.

Another technique is to ask the candidate to do some coding as part of the phone screen. This is where you really want to weed out the folks who can't code, so you don't waste lots of time interviewing them. I haven't tried this, but it would be nice to ask somebody for a 2 hour phone screen. Talk to them for an hour, then give them a simple coding problem and ask them to e-mail you their code in an hour. Obviously you'd want to pick something a bit off the beaten track, so they can't just google a solution.


33
[0] [2009-07-05 18:18:53] Rev316

I'm a not a big fan of writing code during an interview (or performing other tricks to ensure a proper reward of my resume's legitimacy); for many reasons that Uri already mentioned. Although I understand the purpose of this type of interview, I honestly wouldn't blame the candidate at all for failing over something that's already stressful to begin with.

Many "larger-software" recruiters I've talk to (still an undergrad here) volley on this fence. So, to really answer your question as one who might be taking this interview... make it light & enjoyable. There ARE better ways to distinguish a candidate then making him solve a puzzle for half an hour.

IMHO.


34
[0] [2010-09-01 12:50:12] wadesworld

The important thing when watching someone code is not to get wrapped up in small things. It's easy to feel a ton of pressure coding while someone watches. Don't be concerned if they can't remember how to do something very basic.

What you're looking for is the thought process. If they describe what they're trying to do and you can tell that their thought process is leading them the in the right direction, chances are good they wouldn't be making the same small mistakes when you're not watching.

For example, if someone can't remember how to setup junit for a test case, I'm not hugely concerned if they can tell me what a test case does and describe the correct strategy for testing the code in question.


35
[0] [2011-04-24 05:53:38] Mikey

I was once rejected for a job because of exactly that problem - they had me coding and answering questions with a bunch of their programmers sitting around waiting for me to screw up. Told them I don't perform well under such circumstances. I didn't get the job and I look at it as their loss, because I know I could have done very well for them.

Although I worked at a very high pressure job for 12 years, as many have remarked, it's an entirely different type of pressure.

As for the analogy to a musician, it's inappropriate. If someone at an interview asked me write a for loop or a function that returns a string, of course I could do it with my eyes closed, just as a musician can play a piece by heart or off a sheet of music. But how about if you asked the musician to compose something good on the spot with everyone watching? Analyzing a problem, coming up with a solution and then coding it takes clarity of mind as well as trial and error. Hard to do that with someone breathing down your neck who's holding your future in their hands and you have to impress them.


36
[0] [2012-02-21 05:03:50] KnownHinson

Yes, you are reading too much into that reaction. In fact you are also tipping your hand and revealing how immature your organization is.

When it comes to interviewing developers "whiteboard" demonstrations are the last refuge of the unimaginative and frankly the unskilled at interviewing. I've been a developer both salaried and contractor for two decades. I've worked for organizations of all types and sizes and have been on both sides of the interview process numerous times. With just half a dozen targeted technical questions, in normal conversation like civilized human beings, I can tell if a candidate has genuine skills relevant to the hiring position. With another few questions I can tell if the candidate has relevant knowledge surpassing my own, which is almost always what is needed. It's really not that hard to do and those who cannot do it should not be in a position of filtering talent because they will probably not make the best choices.

The reason many people have trouble with whiteboard interviews is because, regardless of how you may try to lower the temperature, it is fundamentally a confrontational process in what is already a stressful situation. Professionals with many years of acquired skills, documented experience and powerful references are essentially told to prove they are not frauds by writing some idiotic five line stringify function in front of a jury of strangers. It's the equivalent of an interviewee demanding a would-be manager to stand at a whiteboard and draw out how the department could be better organized as a condition of making a job acceptance decision. No manager should ever accept a ridiculously one-sided request like that yet it's demanded of the interviewee in what should otherwise be a two-way process of discovery.

If you want to test a candidate's actual coding skills it is a better tactic to provide a written test in advance. A 5 challenge test ranging from what should be simple to what would be challenging even for yourself is sufficient. The reason? It is a truer test of quality of work than a whiteboard since none of us spend a lot of time programming whiteboards. It is especially effective if you inform the candidate that the result will be scrutinized by the same process the shop uses for its own work. Why? Because the candidate will not have the cover of time pressure to explain away sloppy work. If code doesn't compile it's not because shortcuts were made due to a marker running out of ink. In the end you'll be judging the artisan by his self-endorsed best result unaffected by a flawed process.

And if he uses Google, so what? I use Google and various technical sites daily in my own work and I'd be suspicious of the abilities of any other developer who does not. If your existing team cannot craft unique and though-provoking challenges then you've just confirmed that your hiring process needs changing.


37
[-3] [2009-06-30 00:49:46] Igor Krivokon

It's a lame excuse.

Imagine you're hiring a musician for an orchestra. Would you not ask them to play something? How about an excuse like "it's too much pressure playing alone, and when you're watching me so closely", "I need to sit in an orchestra pit, not here in a plain conference room", "How am I supposed to play properly without a tuxedo?"

A good musician will play any tune well, anywhere, anytime. A bad one will come with excuses.


38