Blog Posts

Have you ever wondered how will the machine learning frameworks of the ‘20s look like? In this essay, I examine the directions AI research might take and the requirements they impose on the tools at our disposal, concluding with an overview of what I believe to be the two strong candidates: JAX and S4TF.

CONTINUE READING

In this tutorial, I will give an overview of the TensorFlow 2.x features through the lens of deep reinforcement learning (DRL) by implementing an advantage actor-critic (A2C) agent, solving the classic CartPole-v0 environment. While the goal is to showcase TensorFlow 2.x, I will do my best to make DRL approachable as well, including a birds-eye overview of the field.

CONTINUE READING

After winning against Go world champion Lee Sedol in 2016, DeepMind has announced the next challenge they will focus on: StarCraft II. Today they are finally ready to unveil something at 6PM GMT and with this blog post I want to help both StarCraft players and AI researchers appreciate the scope of what they’re about to experience. I will give a brief overview of the challenge, address some of the common misconceptions, and speculate a bit on what we’ll see.

CONTINUE READING

Undergraduate major is often the first significant career decision a person makes in his life. As artificial intelligence (AI) becomes more and more ingrained in our society, many people begin to consider a career in AI as a viable choice in their life. However, it is still very rare to have an undergraduate degree fully dedicated to AI, so people opt for what they perceive to be the next best thing - computer science. But I believe there is a better alternative: statistics, and in this blog post I will try to explain why, based on my own example.

CONTINUE READING

Projects

Reaver: Modular Deep Reinforcement Learning

Modular DRL framework with a focus on StarCraft II, following in DeepMind’s footsteps, replicating results. Created with performance, extensibility, and reproducibility in mind.
Works with Gym, Atari, and MuJoCo as well.

Starter Agent for Coders Strike Back AI Challenge

Starter bot for an AI programming challenge. Includes fast simulation engine and basic building blocks necessary to implement a competitive bot. Written in C++.

Talks

  • AI & Deep Reinforcement Learning - University of Tartu, 2019
  • Reinforcement Learning Guest Lecture - University of Tartu, 2018
  • Deep Reinforcement Learning - DevClub, Tallinn, 2018
  • Behavior Driven Development with Behat and Mink - DevClub, Tallinn, 2013

Teaching

Teaching Assistant, University of Tartu:

  • LTAT.02.001: Artificial Neural Networks, Spring 2019
  • MTAT.03.317: Deep Reinforcement Learning, Fall 2018