tl;dr Automatic scheduling(timetabling) comes under a special set of mathematical problems named as NP Complete which makes it impossible to generate and validate solutions ‘quickly’. Imagine a magical way of solving a Sudoku puzzle without having to check whether it is correct.

Introduction

Being the CEO of an EdTech startup since 2009, I had come across this challenge multiple times. I was lucky enough to have some brilliant programmers teach me the concept of NP Complete problems in an easy way to understand during the initial years of the startup. As a result I was able to communicate this challenge to non-programmers in layman terms who were most of the times potential clients and partners. Here is the NP Complete problem and automatic timetable generation challenge explained in the simplest way possible.

Constraint satisfaction problems

Constraint satisfaction problems (CSPs) are mathematical problems defined as a set of objects whose state must satisfy a number of constraints or limitations. Famous examples of simple problems that can be modelled as a constraint satisfaction problem include Eight queens puzzle, Map coloring problem, Crossword & Sudoku.

Sudoku is a Constraint satisfaction problem

There are 81 variables. They all have the same range of possible values: integers in [1,9]. The constraints are

All variables in the same row must have unique values.

All variables in the same column must have unique values.

All variables in the same 3*3 subsquare must have unique values.

You can solve the Sudoku puzzle filling one variable at a time according to the rules set by constraints. The main difficulty here is that if you give one variable a value, that removes possible values from other variables that might be necessary for a solution. This demands backtracking, which is why using a pencil and eraser is the best way to solve a Sudoku puzzle.

Can a Sudoku puzzle be solved without backtracking? Can a human or computer solve Sudoku puzzle at one go without crosschecking the constraint satisfaction? There is no clear answer. Researches are still going on. Example: An algorithm has been discovered that is deterministic (i.e. no backtracking), and guaranteed to find a solution to all Sudoku problems but it’s quite complex. http://www.nature.com/articles/srep00725

Now coming back to our topic of automatic timetables, the total timetable for all courses and batches in a school or university is similar to multiple Sudoku puzzles which increases the complexity by adding more constraints of infrastructure, lecturer etc..

If you already think you understood the challenges of automatic scheduling, you are wrong. We are just starting.

Knapsack problem (Optimized resource allocation)

Knapsack problem refers to the commonplace problem of packing your most valuable or useful items without overloading your luggage. It derives its name from the problem faced by someone who is constrained by a fixed-size knapsack(backpack) and must fill it with the most valuable items.

Example of a one-dimensional (constraint) knapsack problem: which boxes should be chosen to maximize the amount of money while still keeping the overall weight under or equal to 15 kg? A multiple constrained problem could consider both the weight and volume of the boxes. Source — https://en.wikipedia.org/wiki/Knapsack_problem

The knapsack problem occurs commonly in role-playing games, both digital and paper-based (prominent examples include The Elder Scrolls series and the Dungeons and Dragons game, respectively), where the player character is constrained by their encumbrance threshold when carrying items and treasure, which regularly forces the player to evaluate the items’ value-to-weight ratio in order to bring only the most value-dense items to a merchant.

Knapsack problem is a good example for an NP Complete problem. You might have noticed the mention in the xkcd comic at the start of this article.

NP Complete problems

In computational complexity theory, a complexity class is a set of problems of related resource-based complexity. There are two classes of problems; P and NP. (there are many more, but we will ignore the rest for this article)

P = Polynomial

NP = Non-deterministic Polynomial

Problems in P are solvable by a polynomial-time algorithm. ex: Multiplication of two numbers. Problems in NP are check-able by a polynomial-time algorithm, but not necessarily solvable. ex: Knapsack problem

If the solution to a problem is easy to check for correctness, is the problem easy to solve?

As of today, the computer scientists know fast algorithms to solve P type problems, but don’t know a fast algorithm to solve the NP type problems. But fast algorithms exist to verify whether a solution to the NP problem is correct.

So NP is the class of problems where, if someone comes up to you and says “here’s the answer”, you can efficiently check whether the answer is right. Think of a solved Sudoku puzzle or solved Knapsack problem or a generated timetable for School. The time required to solve the NP problem using any currently known algorithm increases very quickly as the size of the problem grows. As a consequence, determining whether or not it is possible to solve these problems quickly, called the P versus NP problem, is one of the principal unsolved problems in computer science today. Read more on P vs NP problem

The NP Class is further divided in to NP, NP-Hard and NP-Complete sets of problems. Automatic timetable generation is an NP Complete problem as it is easy to check/verify the correctness of a generated timetable, but the time to generate timetable grow exponentially as the amount of constraints and variables increases.

If scientists discover a fast algorithm for automatic timetable generation, this would be a major discovery because then programmers would be able to use fast algorithms to solve problems that they previously thought could only be solved with slower algorithms. Which means that programmers would be able to make much faster software than they previously thought was possible just by modifying the algorithm alone and not relying on hardware advances.

The challenge with timetabling is not only NP Completeness

NP Completeness is not only what makes true automatic timetable generation difficult. The hardest part of the scheduling problems is the reliability of the collected constraints. Most of the constraints collected upfront to the timetable generation will be based on past data and can change randomly.

Ex: Teacher A, who is also the vice-principal of the school has administrative work in the morning hours which the timetabling office does not know about, attendance is marked in the first hour itself and so the first subject should always be that of the class teacher, teacher B who teaches in Grade 5 during the morning shift also takes classes for Grade 6 on certain days, so on those days all her classes should get over within the 8 hours she is in school etc.

So we need a system where the constraints can be changed last minute without affecting the whole timetable.

If you want to seriously pursue solving the automatic scheduling problem, see http://www.asap.cs.nott.ac.uk/watt/resources/university.html for a list of papers that may help get you started.

I had always marvelled by teachers at their capacity to be patient with the sheer number of diverse students, admired their tenacity and discovered a whole new level of respect for them as they solve this NP problem at the start of every academic year. Their work is as complex, challenging and time consuming than many of the cutting edge technologies.

Fedena’s Automatic Timetable

Our product Fedena is an ERP software for schools which is shipping a beta version of the automatic timetable generator this Christmas. It is not the best yet, but better than the affordable alternatives available in the market. And our road-map is very promising. If you are interested to know more visit our website and get in touch with our team

Note: Trying to simplify complex mathematical concepts is difficult. If you think I have made errors in my explanation, please let me know through your comments. Thank you.