What is the NCLab Project SANDI assessment?

The NCLab Project SANDI Computational Thinking assessment is a 16-step series of tutorials, tasks, and quizzes that inform us about a candidate’s ability to:

• Learn something new.
• Follow directions.
• Use what they learn to build and test a procedure.
• Apply their new knowledge to solve problems.

A Little History

The method used in the assessment is based on one developed at Stanford University around 1980 to teach “computational thinking” to engineering students. Educators quickly realized that the method worked well with children and non-technical people and have used it in many courses over the decades. NCLab has developed the most sophisticated version of the method, described in the next section.

Computational thinking is a term coined in the 1950s to describe how machines think: in other words, what inputs, procedures, and outputs machines need to successfully and reliably carry out a task. As humans, we have a lifetime of experiences that guide how our brains filter information and make decisions. Even sophisticated machines can’t match what we do. Computational thinking builds a bridge.

Nearly all jobs and professions interact with some automated process. This assessment gives us an idea of a candidate’s readiness to work in the modern technological environment.

Since introducing the method in 2013, NCLab has monitored how people learn with it and made many refinements over the years. NCLab developed an assessment based on the method to predict how people would perform in NCLab’s complete training programs and has found it to be an effective screening tool. For Project SANDI, since all current jobs interact with automation and technology, this assessment provides valuable information for a broad range of career outcomes.

How Does the Computational Thinking Assessment Work?  

Working with Karel is like learning how to play chess. A six-year-old can learn the basics, but could still be working on new moves at age ninety. The method is elegantly simple. 

A robot named Karel (a nod to the author who coined the word “robot” in the 1920s) navigates a grid, detecting various objects. The objects are either collectibles, containers, or obstacles. Karel has sensors that detect the type of object. Karel only has five commands: go, left, right, get, and put. 

We use these commands to write procedures for the robot. We can test the procedures as many times as needed to make sure they work. This “iterative” testing procedure is engaging: the results are immediate and people use the feedback to verify, debug, or improve the procedure before submitting it for grading.

This is not just another video tutorial with questions and answers. This is an interactive dive into working with automation.

Assessment overview

There are 16 levels to the assessment, which test four skill areas in computational thinking. Within each skill area, the candidates complete four levels:

Level x.1      Read a tutorial and play an interactive demonstration.

Level x.2       Practice the skill by completing exercises to solve the first graded task.

                     Exercises include hints, templates, and solutions.

                     Graded task #1. 

                     The graded tasks behave like exercises but do not include the solution.

Level x.3      Graded task #2.

Level x.4      Quiz based on skills learned so far. 

                     This quiz tests critical thinking rather than simply testing recall.

What does a candidate learn about and demonstrate in each of the four skill areas? In the assessment, the commands are limited to what the robot can do. This might seem a little abstract.

To understand each skill area, it helps to think of real-life analogies. Here are some examples from a school cafeteria lunch line.

If you created a persona in Module 1, think of a real-life scenario where they might use these skills.
In general, this is how automated procedures work. A candidate might work with such processes in manufacturing assembly lines, data processing, medical procedures, retail ordering, or farming. 

How Does a Candidate Take the Computational Thinking Assessment?

Equipment Requirements

The candidate needs a computer with an up-to-date browser (Edge, Chrome, Firefox, Safari for example) connected to the internet. It cannot be done on a phone. The assessment can be taken at a library, a school or training facility, or at home. All the work is saved on the cloud, so the physical location does not matter.

Bookmark:  https://nclab.com/sandi/

Assessment Code for  Explorers: dbcrs

Setting Up the Assessment

The candidate goes to the NCLab SANDI page 

The instructions for setting up and taking the assessment are on the page. Briefly, the candidate will follow these steps:

1. Fill out and submit a form to create an NCLab account with the first and last name, email address, and assessment code. 

2. Open the email containing the activation link. Click on the link to load the NCLab desktop page.

3. Start working on the assessment.

Taking the Computational Thinking Assessment

NCLab recommends setting aside three hours for quiet, uninterrupted work. It is best to do the assessment in one sitting. If the candidate needs to stop, they can log off and resume progress later. Their progress is automatically saved.

Post-assessment Interview (for Candidates who Complete the Full Assessment)

At the end of the assessment, the candidate schedules a call with an NCLab Intake Specialist from a form that shows available dates and times. This 15-minute interview confirms the candidate’s location and stage in the process and asks three open questions about the assessment. The interview provides anecdotal information about the candidate and their experience with the assessment and verifies that they have taken it themselves.

Results

The candidate will receive a summary report based on the completed online assessment and the interview.

The navigator will receive a full report based on the completed online assessment and the interview. If the navigator has sent a candidate to take the assessment and has not completed it, they can request a partial score.

NCLab uses a scoring system to rate performance on four main factors:

1. Perseverance and completion.
2. Attention to detail and self-monitoring.
3. Independent problem-solving.
4. Critical thinking.

Perseverance and completion: The total weighted score can range from 0 to 100. Anyone completing the whole assessment will score a minimum of 60 points.

Attention to detail and self-monitoring: The candidate can test their procedure and make corrections before submitting it for grading. A candidate who reads the tutorial carefully does all the exercises, troubleshoots their answer, and checks it against the requirements before submitting it to the grader is more likely to pass the grade on the first attempt. A failed attempt still allows the candidate to go back and rework the answer and resubmit as many times as needed to pass. 

Independent problem solving: Because NCLab supplies both hints and templates (fill in the blank answers), we can measure how independently the candidate solves each graded task. Using a hint for a task shows that they need some additional support. Using a template indicates that they depend on a specific outline to solve the task. If they can solve a task without either a hint or a template, that shows independence. Certain jobs may require following procedures with specific steps, whereas others require more independent problem-solving.

Critical thinking: At the end of each cycle, the quizzes require the candidate to think about each possible response rather than simply recalling information in the tutorials. They may, for example, look at different patterns and determine which one works best to solve the problem.

Summary Report Sent to Navigator with a Copy to the Candidate

The summary report provides two aggregate scores, one for problem-solving and one for critical thinking. The problem-solving aggregate score is based on weighted scores on the subtests. The critical thinking score is based on the quiz average score.

The report also contains the answers to three questions posed during the post-assessment interview and a short summary of the call.

See sample assessment report below.

Where does the assessment fit in the SANDI 10 step process?

All Community of Practice members will work to identify and place eligible candidates into a program that leads to completing Step 10: Getting a Career! 

To refresh your memory, here are the ten steps:

The Ten Steps

Ten Steps Graphic

A candidate might arrive at the NCLab SANDI assessment via different avenues.

No matter how people reach https://nclab.com/sandi/, they have arrived at the right place to take the SANDI assessment.

Various paths to the assessment

Project SANDI Step 3 to 5 (Libraries)

As part of Step 3 or Step 4, the library navigator will point the candidate to the NCLab SANDI webpage. The library navigator will receive the report to assist in evaluation in Step 5 and will pass it along with other pertinent information to the preferred training provider. NCLab is excited to see the libraries as partners in this project. We have worked with libraries across the state for many years and are ready to coordinate their efforts to attract and screen candidates.

Project SANDI Step 6 (College Navigators)

The college navigator may have candidates who have not taken the assessment but could benefit from it.

Employment and Career Centers

Since NCLab courses are listed on ETPL, some candidates may already have an interest in NCLab training packages and choose to take the assessment.

Organic traffic to NCLab

People interested in data analytics or programming may find NCLab independently. From an initial call, NCLab might determine that the person should be evaluated for eligibility under the SANDI grant. They may have already taken the assessment before applying for the grant funding.

Regardless of the source, NCLab will share the assessment results with the appropriate navigators in the SANDI program.

IT, but not just for IT

The assessment was initially designed to screen people for the data analytics and programming training programs at NCLab. SANDI organizers envisioned it as a screen for technology-related career training programs. However, there are good reasons to steer a broader group into the assessment.

1. People have hidden talents. Very few assessments test computational skills, and a person may not be aware that they have these abilities. That discovery can lead to a wide range of well-paying career opportunities.

2. Technology is not just for specialists. Everyone has to interact with automation. This assessment helps determine their readiness.

3. The assessment provides how an individual handles self-paced, online learning. This is a new paradigm for many people.

4. Taking the assessment can answer a straightforward question: Can a person learn new things and apply what they learned? In this sense, the assessment is one indicator of success for any program within SANDI.

Your feedback is valuable, let us know how we can improve this page. Click here