The world’s leading publication for data science, AI, and ML professionals.

Can AI Really Help You at Passing Interviews?

Thoughts, tips, and some prompts

Andertoons image license
Andertoons image license

I have been hearing concerns lately from fellow interviewers that they are worried that candidates may leverage AI for passing technical interviews. The concern is that using LLM technology like ChatGPT or Bard, they may "game" the system and solve problems during technical interviews.

I think you should make use of AI for passing interviews, but not how you would initially expect.

The most important element for passing an interview is really preparing for it and the best way to prepare is to practice.

Some companies would charge thousands of dollars and would take candidates through mock interviews, giving them feedback and highlighting areas to improve. This creates a significant opportunity gap for people that can’t afford to pay for such services.

With the emergence of the new LLM technology, the question is if AI is leveling the playing field and if the tech can be used as a tutor for practicing mock interviews. First, let’s dive into the wrong way of leveraging AI.

The wrong way

Let’s first try to address this valid concern. Should you be using an LLM service (could be ChatGPT, Bard – you name it) to try to get the answers to technical questions while in the interview? The tech works, but the real question is should you, as a candidate, really use it that way?

In my opinion, the answer is no. Doing this will hurt you both in the short and long run. Let’s go over some reasons for not doing it:

You will get caught

Interviews are stressful, most likely the interviewer will catch on if you are split screening, trying to search for the answer and saying something you don’t fully understand.

I have been holding over 1000 interviews throughout my career and I have had a few instances where candidates were trying to do a split-screen approach and were trying to search for the answer while talking to me. This was very obvious.

How would I know they were searching for answers? They would not pay attention to my hints, they would freeze and then suddenly start to speak from a different angle.

Interviews are very stressful. Adding more stress on top and worrying that you will get caught will lower your IQ considerably.

The most likely outcome is that you will fail the interview. I have seen this happen a couple of times. You may be very intelligent and hard-working, but the second you cheat, you will not be a cultural add to the team – any team.

Impossible to do in person

Before COVID, a lot of the interviews were in person. Big tech would fly you in for sessions of on-site interviews. With COVID this has changed a bit, but now with the return to office, more and more interviews will take place in person. At this point, if you were invited onsite, using LLMs for solving questions during the interview would be close to impossible.

Gaming the wrong angle

Even if you get away with it and get the job, most likely you will struggle in your new position. You could try to convince yourself that you just need to get in and then you will figure it all out along the way, but you will add a lot of stress on yourself knowing deep down that you cheated the interview.

It’s like starting a relationship with a lie – it most likely won’t end well.


A better way to leverage AI

It looks like LLMs are very good at chat interactions, it’s in the name: large language models. You most likely have experimented with some version of ChatGPT or Bard. If not, I encourage you to give them a try.

From the experiments that folks have been doing so far, it looks like LLMs could potentially be very good tutors given the correct way of prompting. Tutoring and getting access to services from specialized companies to mentor or coach you to learn the skills is pretty expensive, but now with AI things may become free.

Now you could use LLMs, the ChatGPTs, and Bards of today, to perfect your skills and get better at interviewing.

Some Tips and Tricks from the AI Industry

With LLMs a new "discipline" has risen, the so-called prompt engineering which is a clever name for "be clear what you ask your AI LLM model to provide" so you get the answer you need.

There are some gotchas in the industry so far regarding prompt engineering that you may leverage to get better results. Yes, AI can hallucinate, another clever term for going off the rails and giving wrong answers. Here is where good prompt engineering comes into play to reduce the chances of hallucination. I will try to quote different literature so you see I am not making this stuff up and can dig deeper on your own, otherwise, you could just use the prompts.

Context, context, context

You may have heard this a million times, but context matters. It is the same with LLMs. The better you set the context for the LLM engine the better answers you will get.

The step-by-step approach is better

The same applies to interviewing. An interviewer would like to see the steps you have taken to reach a specific result. If you ask in your prompt to produce a step-by-step answer you will have a better chance the model does not hallucinate and you will get more accurate results.

This is also known in the industry as Chain-of-Thought Reasoning (zero-shot CoT). You can achieve this by appending a "let’s think step by step" towards the end of your prompt.

LLMs are good at checking themselves

For some reason, even if an LLM hallucinates and you ask the model if the answer was correct given your initial question, it will usually know it was wrong and give you the correct answer. If they are good at checking their answer, they are also good at checking your answer.

LLMs are not good at math

LLMs don’t do well on basic arithmetics so try to avoid those subjects.

This should be fine unless you want to pass an interview for calculus and basic arithmetics.

But enough with all of these technicalities, let’s see how you could put it to use and incorporate some of the previous industry learnings. Feel free to experiment with your text prompts.


Mock interview script & applied prompt tips

Let’s take a look at how you could turn your LLM chatbot partner into an interviewer that will take you through a mock demonstration. Below is a breakdown of the main sections during a mock interview.

Small talk

This is to put the candidate at ease and describe the process. We don’t need any prompt here or you could have a friendly chat with your LLM about the weather, your choice.

No prompt for now.

Present the problem to the candidate

Here we can use the following prompt to generate some potential interview problems for an interview in Java:

LLM prompt: "You are an interviewer with a FAANG tech company. 
What is a list of algorithmic problems to choose from in order to assess if a candidate is a good technical fit. 
The candidate's preferred language is Java."

ChatGPT example:

image made by the author
image made by the author

Don’t forget, giving context helps. Usually, problems fall into the algorithmic or system design phase, you could choose which type you train for by changing the terms in the prompt.

Now you can choose your problem, let’s say we go for the Palindrome algorithmic problem.

LLM prompt: "Please present the palindrome problem to the candidate."

ChatGPT example:

image made by the author
image made by the author

Work with the candidate on a solution as in the interview

Here you need to open a Google doc (most likely in the remote interview you will have a shared medium with your interviewer for writing your solutions.) You can start drafting your solution and code and think out loud, like in an interview.

If you need more hints you can ask for them

LLM prompt: "Can you give additional hints and examples without giving the solution?"

Times up

When time expires, assess the solution and give candidates valuable feedback. After you have arrived at a solution that you think works, check if it is a valid one. Copy and paste the code in the LLM prompt after the following prompt. Don’t forget to apply the previous tricks and ask for a step-by-step analysis.

LLM prompt: ""Is this a good solution for the Palindrome problem that the candidate has provided? Let's go through it step by step."

We have the following solution to the problem, where we intentionally add 2 mistakes to see if the engine catches it:

  • changing the correct = length(cleanedInput) -1 to length(cleanedInput) -2
  • Changing the correct right – to right ++
public class Palindrome {
public static boolean isPalindrome(String input) {
String cleanedInput = removeSpacesAndPunctuation(input);
cleanedInput = convertToLowercase(cleanedInput);
int left = 0;
int right = cleanedInput.length() - 2;
while (left < right) {
if (cleanedInput.charAt(left) != cleanedInput.charAt(right)) {
return false; // Characters don't match, not a palindrome
}
left++;
right++;
}
return true; // All characters matched, it's a palindrome
}
public static String removeSpacesAndPunctuation(String input) {
StringBuilder cleanedInput = new StringBuilder();
for (char c : input.toCharArray()) {
if (Character.isLetterOrDigit(c)) {
cleanedInput.append(c);
}
}
return cleanedInput.toString();
}
public static String convertToLowercase(String input) {
return input.toLowerCase();
}
public static void main(String[] args) {
String input = "A man, a plan, a canal: Panama";
System.out.println(isPalindrome(input)); // Output: true
}
}

The output:

image made by the author
image made by the author

It is nice we also got some pointers on the mistakes we made.

Remember, a way to reduce the chance of LLM hallucination is to leverage the step-by-step approach or the CoT by asking for a step-by-step analysis.

After learning from the Palindrome example you could move on to another question.

You could also have the LLM check the answer previously provided, just to make sure it was a correct assessment and to go through it step by step.

LLM prompt: Are you sure that you have provided a correct assessment? Let's go through it step by step.

Limitations for LLMs and mock interviews

Even with applying the tips and tricks presented, LLMs may still hallucinate and say that a wrong solution looks good. The best thing is to fact-check them. Take the code and run it in a simulator to see if it outputs the right answers.

One way you can use LLMs without the fear of them giving wrong answers is by coming up with questions asked in interviews. This will be a good starting point, you can then either use the LLM model to solve the question and fact-check it against running the actual code or search for the solution online after you have solved it.

Tools that may help

If you don’t want to experiment with the LLM chat engines yourself, you can leverage pre-existing tools that use voice and chat and that can get you more comfortable with interviews.

Google has something cool for interview prep: interview-warmup


Use AI for learning, not for "gaming"

The purpose of an interview is to assess if you have the necessary skills and if you are a cultural match for the company. Gaming the system will not make you a cultural fit with many companies out there and most likely will not land you the job.

What you could do is leverage AI in getting a competitive advantage by learning faster, smarter, and preparing for interviews. This way you will increase your value, and you will be able to pass any interview and get the position you would like.


Related Articles