We are currently discussing how theme can be uploaded and set live without any human interaction. When themes are uploaded via the upload page the themes are checked with the Theme Check plugin and then the ticket is created. The Theme Check plugin currently does not catch all of the issues.
Are there any blockers stopping us from switching to a fully automated system specifically related to the the Theme Check plugin? This does not need to be checks it can be other things too.
What is the minimum requirement that all themes should fulfil? Does the Theme Check plugin cover all of the these requirements?
The additional checks are not going to cover all of the requirements. The main aspects that we want to concentrate on are security, spam and things that could break the user’s site like PHP errors and non-namespaced functions.
What specific checks do we need?
Here are a few checks that I thought of:
- Check for the settings API or custom setting solutions
- Counting the number of text domains. We can also have a list of text domains that are normally used a frameworks. This would be extending the “TextDomainCheck” class
- Checking for namespaces
- Checking for duplicate themes from spammers
- Checking for PHP errors
With any automated testing that are always false positives. How do we want to handle false positives?
My solution would be to add the theme check results to the created ticket and then a reviewer can just check if it is a false positive.
The Theme Check plugin can be found on GitHub. There are two issues open at the moment that need a Pull Request. We are also looking for people to help write the additional tests.