Deep Learning Indaba 2018 Reflections

[8min read]

During the week of the 9th – 14th of September 2018 I was blessed with the opportunity to attend the Deep Learning Indaba 2018 held in Stellenbosch, South Africa. The Deep Learning Indaba is a week-long conference that “exists to celebrate and strengthen machine learning in Africa”. It is a wonderful gathering of Machine Learning Researchers, Practitioners and Enthusiasts from all walks of life who share in the collective vision that Africans are and will continue to be “critical contributors, owners and shapers of the coming advances in Artificial Intelligence and Machine Learning”.

The theme of this years Indaba was Masakhane – We Build Together. Keep this theme in mind as you read through this post. By the end you will have a better understanding of the significance of the theme as well as (hopefully) a new, refreshed, or affirmed perspective on what it means to be African.

Note: this blog post will not be a technical deep-dive. Rather, this post will cover several key takeaways which I walked away with from the Indaba.

Here’s how the week went…

Day-to-day Walkthrough

Day 1

Entering the main conference hall for the first time, I found myself feeling a bit anxious and overwhelmed by the crowds and the week ahead. This anxiety was quickly dispelled when Nando de Freitas, Principle Scientist and Team Lead at DeepMind, kicked off the event with a welcoming Keynote that left the 500+ attendees inspired, energised and excited about the week ahead. Nando left the audience with several wise words:

  • Where there’s a problem, there’s an opportunity to do something great.
  • Don’t ever say that a problem cannot be solved.
  • Ideas are not born out of the blue – you have to develop a strong sense of curiosity.

He was then followed by Moustapha Cisse, Team Lead of Google AI in Ghana, and Naila Murray, Senior Scientist at Naver Labs in Europe, who covered the Fundamentals of Deep Learning and Convolutional Models respectively. Moustapha humorously (but quite seriously) stressed that as a Data Science practitioner, you only earn the right to use a Deep Learning framework (Tensorflow, Keras, etc) once you’ve gone through the pain of implementing neural networks from scratch at least once. nn

*(Charles H Martin, 2018)

One thing that was quite apparent to me was that all of the speakers (including those I would later come to see) were not only exceptionally knowledgable in their technical fields, but they were excellent communicators who made it easy for audiences to consume knowledge. This is a beautiful skill to have!

Day 2

Day 2 began with an awesome lab session on Convolutional Neural Networks where we got to implement a basic image classifier. This was then followed a technical lecture by Yabedal Fantaye, Junior Research Chair at the African Institute for Mathematical Sciences in South Africa. Yabedal’s lecture showed how probabilistic thinking is at the heart of Machine Learning because Machine Learning attempts to model data under uncertainty. This stats-heavy session brought out some of my old insecurities as stats and I have never quite gotten along.

Kyunghyun Cho, Assistant Professor at New York University and Research scientist at Facebook AI Research, followed Yabedal with a lecture on Recurrent Neural Networks. Mr. Cho went through the mechanics of Recurrent Neural Networks in a humorous but clear fashion, and he opened my mind to just how massive a field Language Modelling is with particularly useful applications which tackle pressing problems such as breaking down educational barriers through automatic language translation. Day 2 concluded with a lab session on Recurrent Neural Networks where we got to implement a Recurrent Neural Network for time-series estimation and a Recurrent Language Model for character level classification.

Day 3

Shakir Mohamed, Research Scientist at DeepMind, kicked off Day 3 with a lecture on Generative Models. His session was focussed on strengthening probabilistic foundations, where he stressed the fact that all Machine Learning practitioners must build ‘Probabilistic Dexterity’.

probability joke

*(Pbworks, 2014)

This session was followed by the Kambule and Maathai Awards Session – an inspirational awards session which recognises excellent machine learning research and application being done by Africans. This session was particularly special because it celebrated African Machine Learning – Machine Learning being used to positively impact African societies. The Kambule and Maathai awards were awarded to Dr. Justine Nasejje and Yasini Musa Ayami. I urge you to see what great work they have done to enhance the African Machine Learning body of knowledge and to uplift their communities. What particularly stood out for me in this session was seeing that the African continent is exploding with talent and has a lot more brewing. Machine Learning can and is being used to uplift communities throughout Africa. You don’t need to be the smartest person or have the smartest idea to make a positive difference. You just need to embed yourself at the heart of the problems which you’re trying to solve.

The remainder of the day was dedicated to Reinforcement Learning which was covered by Katja Hofmann, Researcher at Microsoft Research Cambridge. Katja took us through the foundations of Reinforcement Learning as well how she currently uses the game Minecraft as an experimentation platform to develop intelligent technologies.

Day 4

For attendees wanting to dive deeper into the technicals of certain fields, 3 parallel tracks were hosted in the morning of Day 4. I attended the session titled the ‘Frontiers of Computer Vision’. This session walked through the history of Computer Vision and how it has developed over the years. We were then fortunate to watch presentations by a select few who are using Computer Vision to solve very interesting and complex problems. Two particularly useful use cases stood out for me wherein Computer Vision was being applied to healthcare problems such as Breast Cancer detection and Early Retinal Tissue Damage Detection.

Following the parallel sessions, a thought-provoking panel discussion was held on Democratising Machine Learning on the African Continent. Truth be told, this topic itself needs a separate blog post to explore its depth but I’ll do my best to summarise the conversation. This discussion explored how Machine Learning can, is being, and should be democratised on the African Continent. Key takeaways from this discussion:

  • Access to internet and computing power are the two most important things needed in order to democratise Machine Learning on the African continent.
  • African countries should not be concerned about brain drain. In an increasingly digitally connected world, there should be no reason why you cannot add value to a place remotely.
  • Those with skills and resources need to drive localised education and skills development in their communities to develop talent. Africa must not be afraid of exporting it’s talent to the rest of the world.
  • Africa needs practical Machine Learning – moving beyond ideation to delivery and execution.

This conversation turned out to be very lively and interactive with the panelists tackling some sensitive topical issues, leading me to believe that they truly are thought leaders in this space.

David Silver, a celebrity in the Machine Learning fraternity, delivered a special session on the success stories of Reinforcement Learning. He is a Principle Scientist at DeepMind and was also the lead researcher on the AlphaGo team responsible for defeating the worlds best human Go player in 2016.

Day 4 was concluded with 3 parallel tracks. I attended a session on Generative Models and Healthcare facilitated by Konstantina Palla, Research Scientist at Microsoft Research Cambridge. One thing that stood out for me from this session was how carefully one has to frame a Machine Learning problem in the context of Healthcare. This is because of the sensitive nature of the work being done e.g disease detection and diagnosis. You cannot trust a Machine Learning model blindly in such contexts without truly understanding what your model is doing.

Day 5

In an unfortunate ‘turn’ of events (car accident), I wasn’t able to attend any of the morning sessions on the final day, but I made it just in time to catch the tail end of another Machine Learning celebrity’s talk: Jeff Dean – Senior Fellow, Google AI. Google Brain lead and co-founder. Co-designer and implementor of Tensorflow, MapReduce, BigTable, and Spanner (seriously, what hasn’t this guy done?). At the end of his talk, Jeff Dean was asked “How have you done so much?”, to which he replied (and I’m paraphrasing here) “Find an area you don’t know about which you want to know about. Then find people in that area who you can learn from and work with. Collaborate.” Hearing this I began doing some deeper reflection on the Indaba over the week and two values stood out for me as my biggest takeaways.

Empathy and Humility

Let me reiterate: over 500 people from all over the world gathered together for a week because of a collective vision – Africans are and will continue to be “critical contributors, owners and shapers of the coming advances in Artificial Intelligence and Machine Learning”. Every one of the attendees at the Indaba this year has, in some way or another, taken it upon themselves to embed themselves at the heart of the challenges being faced in their fields of research, places of work, and communities – that’s empathy! With empathy at their core, every attendee came with a thirst for knowledge, knowing that there’s so much out there to be learnt. Everyone was ready to work with and learn from others – That’s humility! Coupled together, empathy and humility form an unbreakable chain of collaboration that can span across the African continent ultimately helping people to live better. That is Masakhane!

Author: Adit Mehta

I am a lover of all things Data and Tech, with a particular interest in Data Science, Artificial Intelligence and Software Development. I am a self-driven and self-taught 'Techie', always looking for opportunities to use Tech for social good. More fundamentally, however, I have a deep-seated love for education and youth empowerment.

4 thoughts on “Deep Learning Indaba 2018 Reflections”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s