While for feature extraction in a video game typically the whole image is used, this is hardly practical for many real world games. Instead, using a smaller game state reducing the dimension of the parameter space to include essential parameters only seems to be a promising approach. In the game of Foosball, a compact and comprehensive game state description consists of the positional shifts and rotations of the figures and the position of the ball over time.
In particular, velocities and accelerations can be derived from consecutive time samples of the game state. In this paper, a figure detection system to determine the game state in Foosball is presented. We capture a dataset containing the rotations of the rods which were measured using accelerometers and the positional shifts were derived using traditional Computer Vision techniques in a laboratory setting.
This dataset is utilized to train Convolutional Neural Network CNN based end-to-end regression models to predict the rotations and shifts of each rod. We present an evaluation of our system using different state-of-the-art CNNs as base architectures for the regression model.
We show that our system is able to predict the game state with high accuracy. By providing data for both black and white teams, the presented system is intended to provide the required data for future developments of Imitation Learning techniques w. State detection based on Computer Vision techniques has been used in the automation of games using Reinforcement Learning, cf. The common way is to take an input image stream, e.
In contrast to video games, the usage of whole images is often not practical when automating real-world games. In this case, using a lower dimension abstraction of the game, for which we employ the term game state, can be advantageous for the DRL training and prediction.
