This past weekend, the Clodfest Hackathon took place and the Hosting team proposed a project to improve the tools.
Thanks to Pascal (@swissspidy), Cesar, and myself (@javiercasares), we have been able to make progress on some elements that had been in the backlog for over 4 years.
An important detail is that all the changes made are backward compatible so that the default configuration should not affect any of the new features, especially considering GitHub Actions.
What has been achieved?
The first step was to include support for Multi-PHP. This way, a user could submit different reports of the same commit but with different versions of PHP installed.
The second step was to create Multi-Environment. This allows a user to establish different servers with their respective configurations and PHP versions. This allows, for example, having an environment for “shared hosting”, another for “vps”, and another for “cloud”. This is in addition to the Multi-PHP from before.
The third step was to support All-Commits. The tool previously only executed the latest available commit. With this new change, a list of the 10 latest commits will be made, and all of them will be processed in case many commits are sent in a short time. Usually, the tool can take between 5 and 30 minutes per process, and sometimes commits that were never tested in all environments could be lost.
All these changes are accompanied by their respective improvements in the PHPUnit Test Reporter, the plugin that collects and displays the information.
From now on, the initial list will be a summary of the different environments and how many are working and failing. Once inside, we can see the list of all environments and PHP combinations. This is the visualization for developers: one commit, many results.
Furthermore, there is the option for different bots (i.e., different providers) to have their list and on the same screen, they can see the list of the latest tests and thus easily detect when or in what change an error occurred. This is the visualization for providers: one provider, many results.
Where can the changes be seen?
They are not yet approved because they need code review, documentation, and validation by more people. The focus of the Hackathon was to include the functionality, and it was done “too quickly,” so there is duplicated code and improvements can be made. It works, yes, but it’s not pretty.
The PRs with the changes are at:
If you want to contribute, please review and test the code to validate that everything works correctly.
Next steps
In addition to giving a code review, looking for possible errors, and proposing some improvements, we also intend to review the documentation on how to install and maintain the system automatically.
In parallel, we will review with the #core team that everything is working fine and that the GitHub Actions do not give errors, even with the possibility of running different tests depending on the PHP version.