Olá! We’d like to express our gratitude to everyone who stopped by the Test Team (or Core:Test) table at this year’s WCEU Contributor Day 🙇. Your ideas, perspectives, and open discussions help foster initiatives critical to testing WordPress. Thank you to everyone who participated!
Participants at the event covered the following topics (some of which were also referred to #core-test in Slack):
Test Report Templates
- In the proposed Test Reports guidelines, clarify how the green checkmark (✅) and red “X” (❌) emoji should be used in reports: expected vs unexpected.
- Break out the report templates into subpages under a main “Test Report” description in the Test Handbook to improve readability.
- Proposal to provide ticket creation templates for Trac issue reporting, similar to Gutenberg Issues (e.g for Bugs vs Enhancements).
@todoInvestigate whether Trac supports pre-populated template options, for initial post and/or comments.
Easier Test Contributions
- Improve/update the Test Handbook guidance for creating a local WordPress environment.
- The desire for ephemeral test environments (no local install needed) to test PRs and patches. Some ideas:
- Create a Core-targeted version of gutenberg.run.
- Would need to support both PRs and individual Trac patch attachments.
- Utilize a service similar to that used for calypso.live.
- Create a Core-targeted version of gutenberg.run.
- Add Test Handbook guidance for applying patches from GitHub or Trac, covering the various “best practice” methods.
- Reiterate the importance of different environment flavors across the test contributor group (Docker/
wp-env, VVV, Laravel Valet, Local, etc). There shouldn’t be a preference for “the best” or “one way” to run/test WordPress, since it should reflect the real-world variation across the WordPress community. - Assign a new
week-in-testcategory to Week in Test posts, for easier filtering of “where to start” in testing (currently grouped under the more genericsummarycategory).
End-to-End (E2E) Testing
- Questions as to where to begin E2E testing in WordPress:
- Referred to Playwright E2E test migration efforts.
- E2E contributors were encouraged to reach out on #core-test and #core-editor for updated information, as well as to participants in previous discussions.
- It was noted that it’s common for E2E tests to fail intermittently, which can confuse and hamper development. This is often attributed to unexpected delays in DOM updates.
- Consider that E2E testing can passively validate accessibility (“a11y”) as a beneficial byproduct.
- Add E2E section to Week in Test to increase awareness of this aspect of WordPress testing.
Tuesday Meetings Time Change
- It was suggested that Tuesday meetings for
<test-chat>and<test-triage>be shifted from 17:00 to 16:00 UTC to allow broader participation from European contributors. Please share your vote or thoughts here.
Props to @boniu91 for peer review of this post.