The Conference on Neural Information Processing Systems (NeurIPS) is one of the most significant machine learning and artificial intelligence conferences in the world, with top academic and business institutions publishing papers and exchanging knowledge. Tooploox researchers are proud to have contributed to the event with two papers published.
The conference is one of the longest-standing AI conferences, initiated in 1986 by the California Institute of Technology and Bell Laboratories. The goal of the event was to provide fertile ground for researchers of various backgrounds, be they from IT, engineering, or biology, to collaborate on the design and delivery of artificial neural networks.
Through the years the conference has evolved into its current state of a meeting focused on machine learning, statistics, and artificial intelligence. Tooploox proudly contributes to world research by publishing papers and delivering them to the public at this conference.
Delivering better predictions with Non-Gaussian Gaussian Processes for Few-Shot Regression
Gaussian processes are applied widely in statistics to describe multiple aspects of reality, from demography to economy and even to asset management. Yet the Gaussian processes are limited by their sole nature – they are applicable only in the situation where normal distribution is observed.
While there are numerous cases where this occurs, this is not applicable in all cases. For example weather, energy consumption, or the stock market are impossible to predict using the Gaussian method. Or at least – it certainly seemed impossible.
The Tooploox research team has delivered a model that delivers a posterior predictive distribution that is applied on a previously delivered Gaussian function. Thus, after applying the Gaussian method on the data which to be the basis of a prediction, the model delivers the probability distribution for predicted value points that are possible to be non-Gaussian due to external factors or the nature of the process itself.
The model does not deliver an exact prediction of the data, but a distribution of probabilities. In effect, the model provides the user with the information that there is an 80% chance of a particular value, 10% of another, and 5% of two others respectively.
Real-Life Applications
The method was initially designed for few-shot regression problems, but it can be applied to various prediction tasks. This method can be used to improve the accuracy of predictions regarding basically anything data-dependent, including energy usage, demand forecasting in retail, or stock market prices.
Read also: Unintuitive Statistical Phenomenon: Regression to the Mean
While the Gaussian underlay of the model delivers a prediction in an established way, the model combines any external data to update the results as delivered by the Gaussian method. To follow the examples shown above – by combining weather forecasting, time-related power consumption data, and upcoming events (World Cup transmission) the model can deliver more accurate predictions on power consumption. By that, the grid operator can manage the supply in a more responsible and informed way.
This research was delivered by Marcin Sendera, Jacek Tabor, Aleksandra Nowak, Andrzej Bedychaj, Massimiliano Patacchiola, Tomasz Trzcinski, Przemysław Spurek and Maciej Zieba. The paper can be found on the NeurIPS papers page.
Reducing energy consumption with Recycling Predictions in Early Exit Neural Networks
The second paper is a joint work completed with the Jagiellonian University of Cracow and it focuses on reducing energy consumption by neural networks. The more complex a task, the more energy required and the more complex the network needs to be. On the other hand, though, there is a huge variety of difficulty even among similar tasks. For example – it is much easier to spot a black dog in the snow than in the high grass of an open field.
Image recognition is done by deep neural networks composed of multiple layers, with each subsequent layer delivering a prediction of what is seen in the image. As the example mentioned above shows, the answer can be pretty straightforward (a black dog in the snow) or challenging (a black dog in high grass, hidden in a bush, but its tail and nose are visible).
In the first example, an accurate prediction can be delivered in an earlier stage. But in the latter, the network does not have enough confidence to deliver information on whether there is a dog, a bush, or a frog in the image. Before Tooploox’s research, the hints from the previous layers were unused by the subsequent layers and the network had to process the whole image again.
The research, as delivered by Tooploox, enriches the neural network with a function that enables the system to reuse the unused predictions to support and enrich the works of the subsequent layers and effectively reduce the computational cost of image classification by up to 50%.
The method was validated using the Image Net benchmark dataset, widely used to test the performance of new approaches and models in image recognition tasks. Also, The Zero Waste approach proved to be effective in Reinforcement Learning, significantly improving the performance of the RL models in Atari Games.
Real-Life Applications
Energy consumption is a challenge for embedded and edge systems, where there is little to no access to cloud computing power. Also, energy consumption is significant when the network is run on a mobile device or IoT appliance.
Our approach makes image classification more widely available for these devices, effectively opening new fields for AI. Also, when applied to larger systems, it can significantly reduce the power consumption of image recognition tasks as done in cloud-based systems.
The ability to use this system in reinforcement learning environments and tasks delivers a huge power consumption reduction in multiple contexts. For example, it can bring both energy savings and performance boosts for autonomous vehicles and their ability to recognize the objects they approach.
This research was delivered by a team consisting of Maciej Wołczyk, Bartosz Wójcik, Klaudia Bałazy, Igor Podolak, Jacek Tabor, Marek Śmieja and Tomasz Trzcinski. Paper can be found on the NIPS papers page.
Summary
Tooploox is an active contributor to the NeurIPS conference, with this year’s edition not being the first. The 2020 edition of NeurIPS accepted a paper from Tooploox that delivered research on Machine Learning in 3D modeling.
