In this blog post I will describe my overall experience during Google Summer of Code 2018 and also give a detailed summary of my project Object Tracking in Julia with JuliaImages under the NumFOCUS organisation.
The Google Summer of Code is an international annual program in which Google awards stipends to all students who successfully complete a requested free and open-source software coding project during the summer.
Julia is a high-level dynamic programming language designed to address the needs of high-performance numerical analysis and computational science, without the typical need of separate compilation to be fast, while also being effective for general-purpose programming, web use or as a specification language.
Julia version 1.0 was released at the recently concluded JuliaCon 2018 held at University College London, UK where I was invited to present a poster on the work that I had done during Google Summer of Code 2018 involving the development of an object tracking package in pure Julia.
My project was titled Object Tracking in Julia. In my GSoC proposal I had proposed to develop an object tracking package for the JuliaImages ecosystem that would house various optical flow and object tracking algorithms including:
Along with the implementation of these algorithms, I had also proposed to add the documentations and demo codes for these algorithms.
The entire project with all the algorithms and documentation is present in the ImageTracking.jl repository under JuliaImages organisation.
| Tasks | Related PRs | Status |
|---|---|---|
| Lucas - Kanade Optical Flow | PR #1 | Merged |
| Haar like features | PR #2 | Merged |
| Farneback Optical Flow | PR #3 | Merged |
| Speed up Lucas - Kanade | PR #7 | Merged |
| Real world test data for optical flow | PR #8 | Open |
| Boosting Tracker | PR #5 | Open |
| MedianFlow Tracker | PR #6 | Open |
| MIL Tracker | PR #10 | Open |
| Docs and Demos | PR #11 | Open |
In my GSoC proposal I had proposed to implement 2 optical flow algorithms and 4 object tracking algorithms. Out of these, I have completed
implementation and tests for both the optical flow algorithms (Lucas - Kanade and Farneback) and have tried to ensure maximum accuracy and speed in
these implementations. The pull requests related to these algorithms have been merged. Only the PR which adds real world testing data using the famous
Yosemite sequence is not yet merged due to the fact that the data storage formats .jld and .jld2 both seem to cause problems on
macOS and win32 due to the current update process of all Julia packages for version 1.0.
I have added the implementation and tests for the haar like features which are used in different trackers like Boosting and MIL. The haar like features PR has also been merged.
When I was drafting my GSoC proposal I did not have an accurate measure of the time that it would have taken me to implement the different tracking algorithms and thus I had proposed to implement 4 tracking algorithms. But as the coding periods passed by and I started implementing these algorithms I realised that I had not alloted sufficient time to each of the tracking algorithms so as to ensure fast code, proper tests, complete documentation, etc. I discussed this with my mentor Dr. Zygmunt L. Szpak and we decided that I should focus on proper implementations and documentation of most of the algorithms rather than trying to add bare minimum implementations of all the algorithms. So I decided to speed up and document the optical flow algorithms and the first 3 tracking algorithms instead of going on to implement the fourth tracker, this is the Tracking Learning Detection framework.
Therefore, till now I have added the Boosting tracker along with tests and docs. The PR corresponding to this is yet to be merged as my mentor and I have been trying to come up with different things and modifications to fasten the algorithm. Till now we have figured that extra allocations being caused when converting a block of an image to a vector inside different loops is what is causing the algorithm to perform at slower speeds. This PR will be merged as soon as we fix this issue.
I have also added the MedianFlow tracker and the MIL tracker. These PRs are also not merged yet due problems related to different Julia packages not working with different versions (Update to 1.0) and also we need to speed up the MedianFlow algorithm by reducing the allocations.
Blog posts summarizing the work done during the entire coding period.
The ImageTracking package currently houses only a few object tracking algorithms but the tracking API and framework allows for addition of any number of new trackers just by specifying their model, sampler, etc.
Future work will involve:
I would like to thank Dr. Zygmunt L. Szpak for mentoring me throughout the project, and for his constant support and guidance.
The ImageTracking package uses algorithms and test data from packages like:
I would also like to thank the entire Julia community for being so awesome and amazingly helpful.