Ensuring that a theme follows all WordPress Theme Directory requirements can be challenging. While theme requirements are important for maintaining a high standard for themes that are distributed through WordPress.org, checking that themes meet these requirements can be time-consuming for reviewers. Additionally, sometimes theme authors aren’t sure whether their theme meets our requirements, before submitting a theme to the directory.
The Theme Review team maintains a number of different tools and datasets to help theme authors. The Theme Check plugin and Theme Sniffer do a good job of informing theme authors of problems with their code but aren’t able to cover some of the tricker aspects of theme reviews, as both tools run static code analysis. As a result, theme authors & Theme Review team members spend countless hours manually testing themes.
If we could combine some of the existing code analysis tools, automate away some of the manual testing and open them up to more development workflows, could we improve theme quality, alleviate pressure on manual testing and speed up the theme review process?
The Theme Review Action project is looking to answer this question.
It’s also worth noting that this project was loosely informed by the Theme Review’s proposed updates to the overall process designed to improve communication during theme reviews. This proof of concept doesn’t intend to be a replacement for that work, but hopefully an extension.
Guiding Principles
The project should:
- Go beyond simply identifying where themes don’t match our guidelines and help improve theme quality
- Leverage existing tooling
- Provide actionable feedback to theme authors
- Be able to run in different contexts
- Continuous development platforms
- Local machines
Proof of Concept
In the last quarter of 2020, we worked on getting something in a state for developers to try, in the hopes of sparking discussion and educating a path forward.
The project has been added to GitHub: https://github.com/WordPress/theme-review-action.
How does it work?
As mentioned in the project’s Readme.md on GitHub, you can test a theme by doing any of the following:
- Navigate to your WordPress theme folder and run
npx wordpress-theme-check-action - Add the Action to your WordPress Theme Repository on GitHub
- Clone the project and run it locally.
Once triggered, the project roughly does the following:
- Creates a development environment using wordpress/env (& docker)
- Installs the latest version of WordPress
- Installs parent theme if applicable
- Imports the test data
- Activates theme
- Tests to make sure theme has minimum required files
- Runs Theme Review Plugin
- Opens a Browser Session programmatically and runs User Interface tests.
- Outputs results of tests
The User Interface tests use the Browser (Puppeteer), mimicking a user, to answer the following questions :
- Does your theme have appropriate:
- Skip links?
- Element focus states?
- Keyboard navigation?
- Does your theme follow other WCAG Accessibility standards?
- Are there any errors in your:
- JavaScript?
- HTML?
- PHP?
- Gutenberg Block Templates?
- Does your theme include prohibited content?
Of course, since this is only a proof of concept, the test coverage isn’t extensive and will require more refinement.
Example 1: Here’s an example of it running locally using npx.
Note, running using npx on windows isn’t working yet :).
Example 2: Here’s an example of its output GitHub:
What’s next?
Can you try it out on your a theme and answer any of the following questions:
- Did you find it easy to use?
- Does the information help you make changes that improve your theme?
- Did you find any bugs?
- Is there anything missing that could be added?
- Although it’s currently a little slow, can you see yourself using it during your development process (or theme review) if it were faster and more complete?
- Is there an alternative approach to consider?
Ready?
Check out https://github.com/WordPress/theme-review-action and read the Readme file for more detailed information on how to use the project. Any issues or feedback can also be logged as issues in the GitHub repository.
Sincere thanks to @poena for the help!