Exposing NLP routines for Job hunters

DOING DATA SCIENCE FROM SCRATCH TASK BY TASK

I am on a mini-series of posts about creating an NLP application to help people improve their resumes. I introduced the core ideas in my initial post, then I showed how to apply a markup to specific keywords, and more recently, I created a shell application using Python with Vue.js. In this post, Exposing NLP routines for Job hunters, we will focus on building out the service. This series provides a resource for job seekers to check keyword matches, defeating the recruiters’ keyword match screening process. Join me on the next episode, modifying the code base, full-stack style, and documenting our progress.

User Story and tasks

In the agile world, that broadly equates to a new user story, and that story has some tasks as follows.

Story: As a Job Seeker, I want to be able to copy&paste a job description and see the keywords/phrases highlighted.

Tasks:

Back-end: add the NLP class to the web-server
Back-end: add a web services route that receives the text, calls the NLP class and returns the keywords
Front-end: add to and increment the UI with a text area, form, and a submit button to allow the user copy&paste the text.
Front-end: add an API call, pass the text passage to the back-end, collect the response and repaint the UI.

Testing our story with the INVEST framework

Independent: Our story should be self-contained and be released without dependency on other features. We will have to modify several script files, but the story is independent.
Negotiable: Our story is tightly focused on the who, what and why but we avoid any implementation details. The UX designer and developer are free to design and build the product.
Valuable: Once we release the story, a simple copy&paste exercise will highlight keywords and phrases in job descriptions. Such a service is highly valued and very manual for most people.
Estimable: If at the earliest stage, I can already see that at most there are five source files to be modified, it is straightforward to size.
Small: If you follow my work, you will know that I practice extreme agile and use a time box of 3–4 hours. I would say that this is a good sized story within my usual time box.
Testable: We already know what the outcome should be – the text should appear with text markups showing the keywords identified by the Natural Language processing algorithm attached to the back-end.

Next we need to define our acceptance tests which help us

Acceptance tests

The user can navigate to a form that allows that user input the job text
The user can copy&paste the job text and press the submit button to perform the analysis
Once the button gets pressed the text passage is sent for processing via the NLP routines, and there is a good response time from the pipeline
The user gets the result and sees the keyword phrases highlighted on their screen.

Perhaps, as a Data Science professional, you have not had exposure to writing features, stories, EPICS, or task level decomposition as part of an Agile team. If you did have experience, you probably know that we are working on the backlog, grooming that, having the conversation with our developers and doing our sizing ahead of iteration planning. As a writer, it is easy for me to take short cuts since I am a scrum team of just me. This section describes a typical backlog grooming and iteration planning discussion. Often one story point is equivalent to one person-days effort. So let us assume this is one story point but that the feature has value to our service consumers such that we are working on a priority, and it will release at the end of the sprint. Since I am a team of one, as in only me, this is an illustration of full-stack development.

Back-end tasks

There are two tasks assigned to me as the back-end developer:-

Back-end: add the NLP class to the web-server
Back-end: add a web services route that receives the text, calls the NLP class and returns the keywords

Adding the NLP class is perhaps the most straightforward since I have my own Python class already defined.

from nlp import nlp
LangServices = nlp()

Adding the web service route is a little more involved

So I created a web route called keywords which allows a HTML post or get operation and that expects to get the data as a json object. Specifically the json object needs to be

{'text': 'the text given by the user as part of the job definition they are reviewing'}

The text value is passed to the NLP class which responds with a list of keywords and those are wrapped in a JSON objected and transmitted to the browser (front-end).

Front-end tasks

There are two tasks assigned to me as the front-end developer:-

Front-end: add to and increment the UI with a text area, form, and a submit button to allow the user copy&paste the text.
Front-end: add an API call, pass the text passage to the back-end, collect the response and repaint the UI.

Building up the user interface required a bit of design and then some HTML coding. First, let us look at the design that I made, then we need to discuss the HTML.

Image by the author of the initial mock-up of the UI changes for the project

As we are very much in prototyping mode, the UI design is kept simple. We have a large text area where the user can copy&paste a text. They can use the Submit button to process the text or the Reset button to clear the text. Vue.js components use a concept called templates and a script. You can see the code in the following gist.

We are using bootstrap as the design system, and the template is primarily involved in painting the form. discussed in my article Semantic Highlighting provides the markup service painting the keywords in Yellow. The template must also have an accompanying , and you can see the code in my gist below.

There are two methods to service the form. ‘OnReset’ for the reset button press and ‘getKeywords’ for the submit button function. Here is the HTML line of code that links the event handlers to the events if you need to see the link.

<b-form @submit="getKeywords" @reset="onReset" v-if="show">

The ‘onReset’ method only serves to wipe the form text and restore the defaults. ‘getKeywords’ is a bit more involved, but the summary is that it makes an API call to the /keywords route, passing the text as a JSON object. The method then registers the promised response, and the node event loop manages to collect the response from the back-end and process it. The list of keywords returned replaces the default list of keywords stored in the ‘queries’ variable. You can see all the default values below. Vue.js is dynamic, with data changes forcing a ‘dom update’.

return {
 queries: ['birds','scatt'],
 description: 'Tropical birds scattered as Drake veered the Jeep',
 form: {
  text: ''
 },
 show: true,
 payload: {'text': ""}
}

With that, I have completed all the four tasks assigned to me as the developer of this story. I ought to be worried about writing test cases and updating any automated testing, but that was not in the scope of this story for now. I would need my tests as a release condition in any dev-ops workflow, leading to automated bug releases and/or codebase updates. All that is left now is the testing.

Testing

There were four test conditions defined as the acceptance criteria for the story:-

The user can navigate to a form that allows that user to input the job text. This test passed because we have a basic layout with a text area and a couple of buttons to operate it.
The user can copy&paste the job text and press the submit button to perform the analysis. Yes, copy&paste functionality is available, and we do have the submit button.
Once the button gets pressed, the text passage goes for processing via the NLP routines, and there is a good response time from the pipeline. You might laugh, but response time is perfect on my development machine, but everything always works in dev!
The user gets the result and sees the keyword phrases highlighted on their screen. Yes, my user(s) get their work; to them, it appears almost real-time, and they see the keywords and phrases highlighted.

As a more detailed test, I went off and found a job description and tried it.

Image by the author showing a test of posting a job description and looking at the keywords being highlighted.

The testing indicates that the story is done and would get accepted by the Product Owner, who also happens to be me!

Closing

In this article, I continued my concept of assisting job seekers by using Natural Language processing routines to speed up the keyword analysis part of any job application. Over many years, I have discussed, with colleagues and friends, how machine learning and AI practitioners need multi-disciplinary skills that empower them to build prototypes and learn from experience quickly. Many have expressed and agreed with this need, and therefore this is my contribution. I hope you enjoyed the complete series, but please feel free to contact me if you had a question or are interested in the series.