Week 13: Perfecting the forecast results

This week signalized the restart of our classes after the Christmas holidays however in an atypical way, as all classes had to be lectured through Zoom meetings, due to the Governments safety measures implemented named - “Semana de contenção”.

In PLNTDIA classes we discussed the Transportation problems as well as the Network Flow problems. We also looked at the vast types of Automatic Planning, for example Project Planning, Occupation Planning, Process Planning, Trajectory Planning, etc...

In the classes of AAUT1IA, the focus was the Apriori algorithm, Support Vector Machine (SVM) and Decision Trees, where examples were shown as well as some exercises done.

With the submission date of the pre-processed data scheduled for the end of this week, the team focused on attempts to improve the results from our dataset. Some tests were made using different approaches to the Support Vector Regression (SVR) as well as making some changes to the data, but with little improvements.

Noticing that there was a highly instable data associated to the recharging of vehicles, we chose to remove them from the dataset as it was a big negative influence on our results and was not the main target of analysis for this challenge. This removal, combined with the use of scikit-learn library allowed a more promising result. To also improve the houses that showed worst consumption forecasting, clustering techniques were applied but with a very little improvement.

Some tests were also conducted considering the community as a whole, namely using Neural Networks and Random Forest Regression. Surprisingly, this approach did not yield any promising results, having a r2 score hovering around 45%. One possible explanation could be that the household each very have intricate consumption habits that could not be well transposed to a global model, as well as 26 households not being enough to make one outlier negligible. Henceforth, the team has abandoned the idea of a community-wide model and will be working on improving the already promising single household models.

Overall, we were able to achieve an approximated value for r2 of 90% for almost all houses in terms of solar energy generation forecasting and an r2 value between 59% and 96% for consumption forecasting, showing just 3 houses with worse forecasting values.

We also started looking into the technique of Particle Swarm Optimization and how we could implement it to help with decision making process with the data that we are getting from our dataset.