Fighting cybersecurity skills gap using new type of solution

• as a candidate — I took part in a huge number of recruitment processes for many different positions in many companies. I was able to look at different ways (or lack them) of verifying my skills and my knowledge,
• as CTO — I had to build two teams (software developers and cybersecurity specialists) and conducted numerous both soft-skills assessments and verified technical knowledge of candidates,
• as a domain expert — I was asked to help in the recruitment processes taking place in the company (but in different departments) and to verify the technical knowledge of candidates,
• as a software developer and cyber security specialist — I created a web application with many vulnerabilities used in the recruitment processes of web app pentesters (task for candidates — find vulnerabilities and write a report),
• as a student — completing engineering and master’s studies at the University of Technology allowed us to learn how knowledge verification is performed in technical fields at universities,
• as a collector of 'certificates' — I collected so far 20 different industry certificates completed with final tests/exams and I had the opportunity to get acquainted with the assessment of knowledge and skills also in this area.

A problem that mainly affects small and medium-sized companies and recruitment agencies. The popular approach used in such cases is to rely on the content of the CV and limit recruitment process to the assessment of soft skills. In the case of companies, a technical assessment part is sometimes added, but conducted by a person who specializes in other areas of IT (e.g. a network administrator interviewing candidates for the position of leader of a newly established cybersecurity department).

A problem that can occur everywhere, both at universities, in recruitment processes and in certification processes. A small pool of questions combined with the lack of a practical part of assessment results in the possibility of skipping this stage by people whose knowledge and skills are very far from expectations. On the Internet you can find not only dumps of questions with answers to certification tests, but also entire descriptions of solutions to practical tasks for certificates with a practical exam, which are based on exactly the same exam tasks for all participants! That’s not all. On the Internet you can also find dumps of questions with answers regarding passing specific classes at specific universities. In the case of some of the companies, you can even find information about the questions that will be asked during job interviews in particular departments.

I am aware that fighting unfair practices is not an easy task, but passing a technical assessment based on memorizing answers to 30–50 questions is another thing. They don’t even fight!

Fortunately, knowledge and practical skills can be effectively assessed. I have seen it with my own eyes many times or have participated in it. I still see space for improvement here, mainly related to significant reductions in costs and time-consuming processes. Replacing an effective evaluation process with another one that is equally effective but cheaper by… several orders of magnitude sounds quite reasonable, doesn’t it?

People with high technical competences who have so far spent a lot of time on such recruitment processes can simply take up much more interesting tasks in their organization.

Now that we have discussed what we are facing with, it is time to present the features that should characterize the solution to these problems and at the same time the features of platform that I have created.

The most prosaic aspect, but as important as the rest. This distinguishes solutions that have gained popularity from those that have never found wider application. If we have two similar products (price and effect), the solution that is more convenient and easier to use has a much greater chance of being chosen. Simplicity should not be confused with triviality, advanced capabilities can still be available if needed!

Discussions about reducing costs by half are exciting when dealing with large amounts and areas where it is difficult to make further savings. I offer a solution that can reduce the cost to the level of 10–15%. If you have an effective evaluation process and you do it yourself, you can calculate how much you spend on it annually in terms of salaries for your domain experts and on hardware infrastructure. You can significantly reduce these expenses, and in addition, your people will be available to you to do something more exciting!

Without automation, it is difficult to obtain the previous two features. Thanks to it, under these few buttons assigning assessments, there are hidden complex algorithms that select in real time right questions from the library of questions, short-lived containers in which code fragments are executed and evaluated, neural networks preparing a short summary of a given approach and so on.

Of course, a report on a given assessment is created fully automatically, and the person who assigned it has access to all the answers provided by participant. The reports also contain a summary including the results for individual categories that appeared in a given assessment, and a short descriptive summary prepared by the neural network.

The remaining features will not convince anyone to use the proposed solution if the competency assessments are not sufficiently reliable. Therefore, I focused on the following aspects:

• uniqueness of questions — questions are always drawn from the database of the library of available questions, currently there are over 45,000 unique questions in the library of more then 50 different topics. I leave it to you to calculate the probability of generating two assessments with 5 exactly the same questions as your homework,
• practical tasks — they can take various forms, the task database includes tasks in which you have to write a fragment of code, remove a vulnerability from a fragment of code (without damaging the functionality itself), read a fragment of code and understand what is happening in it, or read a fragment code and assess whether it contains vulnerabilities. Programming tasks are assessed in a practical way by activating several additional functions that will check whether the solution works and whether there has been any fraud attempt,
• practical questions — i.e. questions that are not easy to answer using a search engine and ChatGPT-style models, but which are relatively easy to answer if you know the topic. Many of such questions have easy to find (on the Internet or using ChatGPT) answers that are … wrong (sic!).

Of course, for full transparency, you can read the report to find out what questions were included in a given assessment and assess for yourself how well they could assess the actual skills of a given person.

I don’t force you to buy anything blindly (I don’t like to buy blindly myself), so I encourage you to create an account and assign yourself a few assessments for the Demo Pentester profile. The assessment time, and therefore the number of questions, is limited to 30 minutes, just like the database of questions available for the Pentester Demo profile, but they may give some insight into the full version of the solution. Moreover during beta you will have access to 3 regular assessments of your choice for free.

If you don’t want to register new account you can still simply use “See the demo!” button, and platform will generate new demo assessment for you. No commitments, no accounts, complete anonymity.

Naturally, creating a universal test for all types of cybersecurity positions would not be the most sensible idea. Due to their diverse functions and responsibilities, many positions focus on very different sets of issues. The obvious idea was to create profiles for the most popular positions. The profile consists of categories, their percentage in the entire assessment and the level of difficulty. Profiles are therefore filters that ensure that the right questions in the right number are sent to a given assessment from the entire database of available questions.

If none of the profiles fit your needs, no worries! You can also assign assessments based on the category sets you choose! It works exactly the same as in the case of Profiles, but you decide which categories will be included in the assessment, what percentage there will be and what level of difficulty should be selected. So you can create unique profiles for assessments, I’m just giving you a tool for that.

If you have broader needs than individual assessments, please visit this page, where you can find information about private instances of the platform that have additional functionalities, such as: creating your own profiles, recruitment pipelines, managing large numbers of candidates, creating your own questions and categories and other advanced mechanisms.

A new solution — built from scratch (because I could), with new features as a result. Why I built something new instead of using/adapting an existing solution:

• because I could — because I like to create and I enjoy writing code!
• because I wanted to create something that would have specific (mentioned earlier) capabilities...
• ... and only these capabilities. At the same time, I wanted to avoid the overhead resulting from having unnecessary functionalities, which would be a side effect of adapting the existing solution to my needs.
• in some cases, building something new is much easier than adapting something that already exists to a new role.

Cyber Proficiency Center is intended to help solve certain problems or improve existing processes in companies and institutions. You can always propose alternatives to each solution, or at least point out some weaknesses. Naturally, you can be judged by the CV and certificates you have. What if the person is at the beginning of their career? What about people who do not hold strictly security positions but need to have certain knowledge? You can use solutions of other type (e.g. cyber range), but they are not tailored to these needs — at the same time, they have functions that are completely useless here, but they also do not have other functionalities that would be useful. It is difficult to use them for a multidimensional assessment that would take 1–2 hours.

I don’t believe that you can hire someone solely because of their skills — that is, solely based on the assessment report in the Cyber Proficiency Center. On the contrary, soft skills and attitude are very important. It’s simply about being able to perform a substantive assessment (which in many cases may be missing in recruitment processes right now) in a convenient, effective and relatively cheap way. Moreover, not all positions can be filled only by people with good future prospects. Building an effective team means skillfully balancing the best players, craftsmen and promising youngsters. After all, someone has to guarantee the team’s effectiveness and be a source of knowledge for people with less experience.

In the case of recruitment agencies, I have heard the opinion that candidates are assessed substantively by the companies that commissioned the recruitment processes. It’s hard to argue with that and at the same time it makes sense from a business point of view! Nevertheless, adding an initial stage of substantive assessment will be useful in the case of recruitment for positions with relatively little required experience. If we have a lot of candidates whose CVs do not allow us to clearly obtain conclusions about their competences, in such a scenario we can definitely rely on cyber proficiency assessments. And finally, you can reduce the chances of presenting the client with profiles of people who will be rejected by him.

I want to emphasize that I do not believe that Cyber Proficiency Center will replace the entire recruitment process. As I mentioned earlier, the assessment of soft skills and predispositions (ability to develop) is extremely important. Cyber Proficiency Center is simply intended to help improve recruitment processes, providing the possibility of effective technical assessments where it may have been lacking so far, or simply allowing for cost reduction.

While working on Cyber Proficiency Center, I identified more areas where completely new types of solutions can be created that will reduce the skills gap effect in cybersecurity. It will be no secret that after Cyber Proficiency Center becomes popular, I will start building another of these projects. I intend to successively fight the various problems caused by the skills gap in cybersecurity. This problem will probably never completely disappear, but I am convinced that it can be significantly reduced with right tools.