Outline

• Past: Background Story
• Past: The First Version (Jul 11-Apr 13)
  First TEG Grant (~2K SGD, 1 year)
• Past: The Second Version (Jun 13-Apr 15)
  Second TEG Grant (~8K SGD, 2 years
  then 1 year+ hiatus)
• This AY: The Laravel Version (Aug 16-Nov 17)
• Present: World Beater Today?
• Future: 'Virtual Steven'
  1 FYP Students in Sem 1 AY 2017/18

This lecture is a chance for Steven to summarize what he (1) and his team (23 others) have done from ~Jul 2011-present (6 years old as of 02 April 2017) in creating, developing, and maintaining VisuAlgo

Some of these success (and a few failure) stories may be relevant for your own web projects now/in the future

The
Background
Story

Acquiring the Knowledge

1. SoC UG story (2000-04), algo-mania, took Dr Alan Cheng's CS3241 (Comp Graphics), but there was no module like CS3226 back then
2. SoC PhD (2004-09+), I did research on advanced-but-rarely-used algorithms, also using visualization :O
3. My brother (Dr Felix Halim) influence
   He is now a senior software engineer at Google MV

The Motivation

1. SoC TA & Instructor (2007-2010), that boredom of teaching the same (basic) thing repeatedly...
2. SG IOI team leader (2009-present), NUS ACM ICPC coach (2008-present); Realizing the 'non-existence' gap back then (e.g. Binary Indexed Tree, Max Flow, Suffix Array, etc)

The Opportunity

1. Jan-Aug 2010: I finished my PhD, so I decided to wrote the first edition of Competitive Programming textbook (available for free today)
2. SoC Lecturer (2011-2015, SL since 01 Jan 2016): Promoted to Lecturer so I need to supervise some FYP projects...
3. First initial discussion with the first developer Koh Zi Chun during IOI 2011 @ Thailand

The First Version

Supported by CDTL TEG 2011/2012 (~2K SGD)
Presented on Tuesday, 23 April 2013

No longer maintained now :O
Because the better version exists :)

Key Technological Decisions

We were a 'new player' back in 2011

There were other algorithm visualization websites/webpages out there...

1. Use the latest tool out there

We use modern* client-side technologies:
HTML5, CSS3, JS+jQuery for highly interactive visualization

2. Accept any valid inputs

We really code those algorithms in JS
and accept any valid inputs

3. Extensibility

The visualization pages are designed
with future extension in mind

The 'first version' of our own graph drawing library was built for this purpose

4. Be the First*

We include a few advanced graph and
geometry algorithms right from the start

They did not exist in the Internet before year 2011

The First Users

1. CS3233 (Jan-Apr 2012, beta)
2. CS2010 (Aug-Nov 2012, launch)
3. IOI conference @ Italy, Sep 2012
4. CS3233 (Jan-Apr 2013, further refinements)
5. Readers of my Competitive Programming
   textbook, 3rd edition (launched 24 May 2013)

The Initial Reception

It was something along this line:

“It is good, but not world beater yet...”

Why is that so?

During CS2010, Aug-Nov 2012 version, we found out that CS2010 students that year only use the visualization ~twice:
around lecture time and just before test

And that's all :(

The Second Version

Supported by the second TEG Grant (~8K SGD, 2 years)

1. UI/UX Major Upgrade

We switch from B/W, non-responsive website to a
colorful, responsive, more user-friendly website

2. Switch to D3.js

We switch from the reasonably good HTML5 canvas to
an even better HTML5 SVG in D3.js (in our custom library)

3. Yet More Visualizations

Major boost via second CDTL Teaching Enhancement Grant

4. Most Importantly...

The Online Quiz Feature

Motivation - Tutorials

Motivation - Assessments

(Early) Source of Q Bank

I used my collection of pas
data structures and algorithms exams @ NUS 2003-2010
and my own CS2010 exams (2011-2016 :O)

Some (but not all) of these questions can be automated

I will add more new creative randomized questions soon for CS2040/C, CS3233, and CS4324 :)

Randomized Questions

Static questions = fixed answers, e.g. try this
(only the fixed options are permuted)

Issue: Memorizing the answers versus
understanding how to derive the solution

We need to build a system that can
generate questions with randomized parameters

Instant Grading and Feedback

As the questions are randomly generated,
we cannot hardcode the answers

Real algorithms verify the answers (note: there can be ≥1 valid answers) for all questions instantly behind the scene

We give (visual) instant feedback if the answer is wrong

This way, the student can always re-study the visualizations, re-generate new random questions, re-train, and so on until they achieve self mastery of the basic topics

Layman Example

Given A and B, compute A+B?
We randomize A and B within reasonable parameter ranges
Easy: A, B ∈ [0..9], Medium: A, B ∈ [0..99], Hard: A, B ∈ [-99..99]

Suppose that the question generated is:
Given A=5 and B=2, compute A+B?
and student X answers 7 and student Y answers 3

We congratulate X but give feedback to Y saying that he/she may accidentally do subtraction instead of addition

CS2010 Example

Given a random directed weighted graph (from MySQL db),
compute the shortest path from a random source vertex s
to a random destination vertex t?
Easy: Small Graph; Medium: Medium Graph; Hard: Graph with negative weight edge(s)/cycle(s)

If student X/Y answers correctly/wrongly,
We congratulate X but give feedback to Y by showing the visualization of the appropriate Shortest Path algorithm on the same randomized directed weighted graph G
This check is done by a back-end PHP script that really runs a Bellman Ford's algorithm

Server-Side Technologies

So that students cannot run the algorithms
to generate the answers on their own computers

Live Demo (on temporary URL)

Limitations of VA Online Quiz

It is currently impossible to replicate a creative lecturer's brain with a computerized system

Basically, it cannot generate a new question type
that has not been added by the developer
So, clever and diligent NUS students can ace the Online Quiz

It is NOT for Final (Summative) Assessment level yet...
But more for 'Formative Assessment' level at the moment
(read about the pedagogical jargon here)

Not-so-Good Hard Random Q

Too easy to make careless mistake

Better Hard Random Q

Exam-only question and/or parameter range :)

The Laravel Version

This AY 2016/17, 3 of my current FYP students proposed a radical idea at the beginning of Sem 1...

Migrate to Laravel...

1. Laravel Blade Templates

To ensure that the home page, every single visualization page (there are ~23 of them), and all other relevant pages have 100% similar look and feel (use the same CSS/fonts) and load the same set of latest libraries (jQuery, D3.js, etc)

2. Laravel Auth/User Accounts

Previously (actually up to now), anyone on earth can use VisuAlgo anonimously, for free...

But that way, we cannot differentiate user (early/medium/advanced CS student or even CS algorithm instructor :O) authorization levels and everyone will see the same version of VisuAlgo...

Current user account features: https :), /login, /accesscontrol, /profile, /training...

3. A Working I18N/L10N System

It was Steven's dream since ~2014 but only realised in 2017...

Preview of /variables, /translation, /accesscontrol (again)

Performance issue and Laravel's Redis (in-memory database) support to the rescue...

4. 'Virtual Steven'

Life question: What happened if I die today?

What if I can script my (algorithm) lectures as web-based presentations?, preview of /electure

More stuffs that I will only say verbally...

5. Another 'First' Visualizations

New visualizations that are not found elsewhere in the Internet at the moment:

1. Minimum Vertex Cover (MVC) and its related Maximum Independent Set (MIS) NP-hard problem visualization, /mvc
2. Traveling Salesman Problem (TSP) NP-hard problem visualization, /tsp

World Beater Today?

Maybe... Let's see :O...

Search These Keywords

Using your favorite search engine
(Google, Yahoo, Bing, Yandex, etc)

1. Data Structure & Algorithm Visualization
2. Data Structure & Algorithm Animation
3. Algorithm Visualization
4. Data Structure Visualization
5. Algorithm Animation
6. Data Structure Animation
7. [obscure algorithm name] Visualization
   try 'Binary Indexed Tree Visualization',
   try '2-SAT Visualization'

Search These Keywords (Bonus)

Search: '[obscure algorithm name in your native language] visualization [also in your native language]'

Try 'Visualisasi Pohon Biner Terurut'
(the Indonesian translation of
'Binary Search Tree Visualization')

Now Be Amazed...

Search: '[any algorithm name that you can remember]'
That is, drop the keyword 'visualization' or 'animation' :O

Try 'sorting', 'binary heap', 'suffix array', etc

Google Analytics Data - Apr 2016

Average: ~3000+ sessions/day with weekly study pattern...

Google Analytics Data - Apr 2017

Average: ~3200+ (goes up... but only by a bit :O) sessions/day still with weekly study pattern...

GA Data - Last 6 weeks vs
Last 7-12 weeks

Is this going to be the new trend?

Global Reach

The traffic does NOT just come from CS1020+/CS2010+/CS3233/CS4234 (notice the average session duration is much higher in Singapore, due to NUS students usage...)

Global Reach (Chinese)

There are lots of Chinese users, still the key market... (notice the average session duration is on average ~2m across all languages, so language is not (yet) the key factor for higher engagement...)

Other Metrics

Alexa web ranking
(still very far from rank 1, but already OK in its field and much better than 2016 ranking)

Google Search Console
(haven't really spend time and analyze)