As you all know, theme review is one of the things we at the Theme Review Team are doing a lot. We are doing other things, but taking care of the themes repository and checking the quality of themes at wordpress.org has been what this team has been doing for a long long time.
And through this long time period, we noticed several things:
- People don’t like to read requirements or handbooks 🙁
- Some of the issues that are popping up are repetitive and could be caught automatically
- Reviewing themes in trac is really cumbersome
This post is not about the first two things. We are constantly discussing about those on the meetings and in the slack channel.
What I want to discuss, and what the representatives have had a discussion on is the third point. Trac. For those of you who don’t know what trac is, it’s an issue tracking system for software development projects. It’s what the entire WordPress ecosystem uses for handling issues and getting features and bugfixes solved. And it’s been doing this job very well for the past 15 years.
However it’s not perfect. Especially when it comes to code review and theme review. Looking at code changes is difficult, integrating with trac and the wordpress.org is also tricky and limited, and all in all is not the perfect fit for themes.
Is there a solution for this issue?
In order to try to find the solution to this problem, I’ve had an idea to try to use GitHub for code review. It’s a platform that I’ve been doing all my professional reviews on for the past 4 years. It’s also a platform that many open source projects are built on.
Even core started using GitHub for development more and more (it still uses SVN under the hood), and you can also link your wordpress.org account with that on GitHub.
With the pull request review style of reviewing you have a better overview of the code changes, you can check only the changes from the last commit (which also speeds up code review a ton), you can utilize Travis CI to run some automated checks on your code, and you can use GitHub actions to do some tasks when things get approved.
The tagging system is better and clearer, multiple reviewers can review a theme, you can comment directly on the offending line so that the author can see the issue more clearly (and you can leave a suggestion of how to fix the issue). It’s an overall better fit for theme reviews.
Proposed GitHub review flow
So how would this review flow look like? Something like this
You can right click on the image and open it in the new tab to see it enlarged.
The process would still involve trac, as this is something that is working and is ingrained in how themes are added to the repository. The first step wouldn’t change. You’d upload a zip file of your theme, as you do now. The theme is checked using theme-check plugin, a corresponding trac ticket is created.
There will be a repository on GitHub called (this is still not implemented of course) theme-reviews. Let’s say your theme name is uploaded and a trac ticket 12345 - Cool Theme is created. Once the theme is uploaded two branches would be created in the repository out of the master branch: 12345-Cool-Theme and approved-12345-Cool-Theme. A pull request would be created from 12345-Cool-Theme towards the approved-12345-Cool-Theme (both are branched out from master, but the code from the zip file would be added to the 12345-Cool-Theme branch).
The pull request would trigger a Travis CI process where we could run some automated checks like PHPCS checks with WPThemeReview coding standards, StyleLint checks for css or scss files and ESLint for JavaScript code.
These checks could be set to either output info or warnings (we currently have a lot of false positives on some PHPCS checks so we cannot just use those as is), and they could be used as a way to inform authors about potential issues in their themes.
Authors should fork the repo to their repositories and work on their theme branch, so that they can update the code as the review progresses.
Once all the issues are fixed, and the theme is reviewed, a pull request would be merged in the approved-12345-Cool-Theme branch and a GitHub action would be triggered that would tag this as a new version, upload it to Trac and then the theme could be set live. The theme branches would be deleted to keep the repository clean enough.
The above is a proposal of a potential workflow that could help reviewers review code. Organisation in GitHub can have different teams set, and users could be added as reviewers, trusted reviewers and admins. Every group can have certain privileges (approving, merging a PR etc.).
So what’s next?
What is needed now is to dissect this idea – see what steps are needed to happen from a technical standpoint and if this is doable.
We do depend on systems (meta team) help with this one, to see if and how difficult or easy it would be to create branches from the servers that are handling the theme uploads. We need to see what can be done with GitHub actions (we know that there already is a lot that can be done with them – 10up GitHub plugin actions).
We also need advice to see if we missed some steps, and what could be improved.
Ultimately, we want to improve the review experience. GitHub has proven as a great tool to do code reviews – the UI is better than trac and the UX is better, and we can utilise some automatic tools to speed up the reviews. Plus with the core using GitHub more and more (especially with the core editor), it’s about time for the theme review to utilise this great tool in order to improve the reviews and potentially shorten the author wait time.