Why am I building Potterverse?

September 10, 2018

Potterverse is meant to be a platform for Potterheads like me to discover something new, something interesting about the Wizarding world of Harry Potter. It will be powered by some sophisticated algorithms, Machine Learning and Natural Language Processing, to bring out details, facts and theories that are buried deep within.

How and when?

I have always been high on the Wizarding world of Harry Potter and have loved every bit of it. It has been my hobby, rather passion, to collect and note every single detail, questionnaire, theory, video that has been made on Potterverse in a document.

One lazy weekend I thought of dumping some of this content in a search engine and see if I could find something; and being a coder I scrapped and dumped the data in Elasticsearch and Voila! I had my very own search engine that could search anything from my favourite Wizarding world. While I was writing the code for it, various ideas and features came to my mind and one thing led to another and at the end I had a rough list of 30 odd features that I could build specific to Potterverse … and thus Potterverse was born.


It was the year 2014 when I created my first search engine on top of 43 GB of Wikipedia data, and since then I had an inclination towards information retrieval and data mining in particular. But in past few years due to some or the other reasons I was unable to pay enough attention to it.

In past few months, I again started playing around with lots of text data, started reading literature about Information Retrieval, Machine Learning, Data Mining and Natural Language Processing, and thought of applying all of my learnings in a platform that is dedicated for a specific domain - and I chose my favourite Wizarding world.

The dataset I chose is from Wikia which is, in all way, ideal for basic experimentation needs, as it is not too Big but at the same time not too small either. The dataset is almost complete, has enough variations to play with and the output can be very easily judged for relevance and usefulness.

In the future, apart from the dataset of Wikia, I would start using some Open datasets available, Videos on YouTube, some Websites like Pottermore, Reddit Threads, etc and make them discoverable at one place and make this platform - a one stop - for all potterheads to discover something pottery.

Last but not the least, I have learnt so much from the Harry Potter community and this platform is my way of giving back. I really hope Potterheads will find this platform useful.