Monday, October 2, 2017

Interview questions for QA analysts

Hiring a QA engineer? Here are some ideas.

We were asked to provide feedback on what I would like to see in a potential QA analyst. Apart from the traditional and essential HR screening (which they know way better than us) this post is about what I would like to see in a QA engineer working in my team.

Conceptual / Basic Questions

Before everything, we're interested in knowing if the person has solid understanding on why that work is being done, why it's important and how to get good results out of it. So, we would like to know, for example:
  • In your understanding, what are the benefits of QA?
  • In your opinion, what is the role of a QA analyst in a project development?
  • What is severity and priority of a bug? Give examples;
  • Define bug triage;
  • Expect some sort of calculation using severity, number of tests, dependent tasks, etc;
  • How to Estimate Testing effort;

Define why do we test software?

Expect the answer to include thinks like:
  • the process of assuring that the product being developed is meeting all requirements.
  • the reason to perform testing is to find bugs and make sure that they get fixed.

Why do we do QA?

Expect the answer to include thinks like:
  • to find the bugs before the product is released to customer.
  • to improve the quality of the product
  • to evaluate that the product is according to requirements

When is it the best moment to start QA in a project?

Expect the answer to include thinks like:
  • A good time to start the QA is from the beginning of the project startup.
  • This will lead to plan the process which will make sure that product coming out meets the customer quality expectation.
  • QA also plays a major role in the communication between teams. It gives time to step up the testing environment.
  • The testing phase starts after the test plans are written, reviewed and approved.

What are the key challenges of software testing?

Expect the answer to include thinks like:
  • Application should be stable enough to be tested.
  • Testing always under time constraint
  • Understanding requirements, Domain knowledge and business user perspective understanding
  • Which tests to execute first?
  • Testing the Complete Application
  • Regression testing
  • Lack of skilled testers.
  • Changing requirements
  • Lack of resources, tools and training

Define a testing lifecycle?

There is no standard testing life cycle, but I like to have following phases and would expect some (if not all) of them being mentioned:
  • Test Planning (Test Strategy, Test Plan, Test Bed Creation)
  • Test Development (Test Procedures, Test Scenarios, Test Cases)
  • Test Execution
  • Result Analysis (compare Expected to Actual results)
  • Defect Tracking
  • Reporting

List different types of tests

I expect at least 5 of the following:
  • Manual testing
  • Smoke testing
  • Regression testing
  • Automated testing
  • Stress Test
  • Load Test
  • Performance Test
  • Exploratory Testing

Programming Skills

For me, developers (as sysadmins/devops engineers), would do their job way more efficiently if they knew how to programm. If she knows how to program, then I would ask:
  • to write a simple function that would, for example, write a string backwards;
  • if the person knows html, css? Exercise that a little;
  • does the person know how to hit a restful api to use browserstack, for example;
  • does the person know javascript;
  • does the person know some other scriptting language so he/she would be able to write some automation;

SQL / NoSQL Questions

I would also very interested in knowing if the person knows SQL. Not necessarily super well but well enough to deal with basic sql instructions and be independent. Also, in case you use NoSQl, ability to query your db in your query language like for example, MongoDb's query syntax or Lucene.

Software Automation Questions

In order to maximize the return of a QA in the team, test automation is a must. That said, the person needs to have enough programming skills write reproducible tests using selenium for example. Personally, I would like to know if the person: 
  • thinks automation is important?
  • knows when to automate?
  • knows what are the benefits of automating?
  • knows what are the downsides of automating?
  • have written automated tests?
  • have used selenium?
Having already working selenium is a big advantage for me.

Infrastructure Questions

Expect the answer to include thinks like:
  • What's the difference between a build and a release?
  • Yes, I have used Azure or AWS before;
  • Yes, I know how to use ftp;
  • Yes, I know how to you check if a server is up;
  • have previously used git, github?
  • have previously you/written a wiki?
  • have previously used Firebug / Chrome Dev tools 
  • is able to inspect, communicate js/html issues;

Board/Offline Exercise

While I don't think we should always use the board, it would be interesting to use the board to check how the person would improvise on two scenarios:

Scenario 1 - Write a test document based on a real use case

In this scenario, I would present a simple use case such as how to create an account on my system (please, use a simple enough for your own system):
  1. Navigate to the landing page of my site;
  2. Click on the Register button;
  3. Enter essential information to create account;
Then, ask user to:
  • List what he/she would have validated?
  • What boundary tests could be written?
  • Ask user to write test matrix;

Scenario 2 - Write a bug report

Still on the create account use case, let's assume that the system should email the new user a confirmation token but it didn't. I then would be interested in:
  • How would the person write a report in case the email didn't reach your inbox? 
  • If the person would document simple stuff as:
    • Date of the test, url, browser version, etc;
    • Steps:
      • clicked on button X;
      • Entered the following info...;
      • Clicked submit;
    • Expected result;
    • Actual result;
That is what I, as a developer, would like to see in a basic bug reported: enough information to reproduce and fix the test. Less than that may be insufficient to reproduce it meaning the probably   the candidate is not a good fit.

Differential

I consider the following, differential to have:
  • Selenium/Automation experience;
  • Cloud experience;
  • Being able to discuss technical details with the developer
  • Linux/MacOS experience;

Conclusion

So, this is all that came to my mind. There's probably more to it but I think that the list above covers a good, very strong candidate. Note that this is by no means a checklist but how I describe a very strong QA candidate and I would be very happy to have someone with the above skills in my team.
Posted by
Labels: , , , , , ,

About the Author

Bruno Hildenbrand