This article focuses on a specific issue – symbol/motion detection inside mobile applications. The application area of this technology is quite broad – starting from fitness apps like step counter (pedometers)/count swimming intensity for specific watches and measuring other activities. Also the application area can be other than sport and fitness. For example, tracking driver activity (how many turns he is doing) or automatic recognition of sign language.

The entry point for me was to find a correct technology. I started with reading articles about gesture recognition and tried sensor fusion on Android, with project on github: https://github.com/apacha/sensor-fusion-demo. This application source code can help you easily start to log data, but you can also start logging data just from ready to go apps on google play like this one: https://play.google.com/store/apps/details?id=com.hiq.sensor&hl=en.

Next step was to find an existing way of building a Model which will get a sequence of data from sensor fusion and will have an answer whether specific Gesture exists in this sequence of data or not. This looks like a typical task for machine learning pattern recognition problem. After looking for algorithms, I decided to try either neural networks (because TensorFlow is so mature and available on different platforms) or support vector machine (because I found a project on github that was very similar to my task: https://github.com/imthexie/GestureLearner).

I ended up using SVM as it was very fast. So what I did is compiled GestureLearner project, changed a bit how training data is collected according to my needs – collecting and marking data at the same time with positive and negative values for particular symbol. Based on 300 inputs of positive and negative data, I built an SVM module and tried a detection. To try it on a real device, I built a service that runs every 100 milliseconds and verifies model with data from last 2 seconds (same duration as data vectors for model creation). With the first run of this mechanism I got good results but there were still too many False Positives. So I trained model with these obvious false positives and got good results on my device which satisfied my initial goal.

After that we continued improving model in beta testing on different devices, asking beta testers to collect data of false positives. we have added a special logger for beta test, that collected data for past 20 seconds, so that beta tester had a time to take logs from these 20 seconds when a false positive has happened. We also marked all 10 pieces of splitted data as false to the model.

We got very good results after some time in the model, with about 99% of successful results and very rear false positives, so that we could use it in our specific task. The good thing about using the SVM algorithm is that it is so fast, that we can even run it every 10 milliseconds against the data vector on normal android device. That means that we had quite good reserve (like 10-20 times) for doing other things in parallel.