The AI Contributor weekly meeting reconvened today (Slack archive) to discuss progress, upcoming milestones, and coordination efforts across the projects maintained by the WordPress AI team. This meeting focused on preparation for WordPress 7.0 Beta 1 and tooling improvements for the team.
WordPress 7.0 Beta 1 preparation and timelines
@jeffpaul opened the meeting by reminding the team that WordPress 7.0 Beta 1 is just four weeks away (February 19th). This timeline constraint was emphasized throughout the discussion as a key factor for planning and decision-making.
Key items identified for the 7.0 milestone:
- WP AI Client: a PR or a set of PRs need to be prepared, paired with a merge proposal post on Make/Core
- Abilities API: The client-side JS counterpart is intended to ship as part of 7.0
- Additional core abilities: @jorgefilipecosta expressed hope to include settings management and user management abilities, with post management abilities also under consideration
- Versionless Ability Deprecation: @justlevine advocated for including versionless deprecation and a querying API in 7.0, acknowledging the time constraints
Regarding Gutenberg workflows, @jorgefilipecosta clarified that they remain in early exploratory stages and will not impact the AI team’s efforts. Once shipped, they will simply be another consumer of abilities, with no expected major impact on AI initiatives. The potential for hybrid abilities was acknowledged but remains on hold pending a concrete use case.
WP AI Client updates
@jason_the_adams provided an update on the WP AI Client roadmap:
- Integrating PHP AI Client v0.4.0 into the WP AI Client
- Planning PHP AI Client v0.4.1 to address a bug before the 7.0 milestone
- @flixos90 is exploring the best approach to include PHP AI Client in core, similar to how the Requests library is integrated
- @flixos90 will circle back with details on the integration approach
- @jason_the_adams mentioned a forthcoming PR that will add a filter to control user access to running AI prompts, allowing site administrators to limit access by user, context, and other criteria
- @isotropic to write the merge proposal for the WP AI Client inclusion in WordPress 7.0.
WP Bench and benchmarking tooling
@jeffpaul introduced discussion around WP Bench, an AI benchmarking tool developed by the team. @jason_the_adams provided an overview:
- WP Bench evaluates various language models against WordPress-specific challenges, providing metrics on model performance in terms of WordPress knowledge and code execution
- The goal is to motivate AI providers to improve their models for WordPress contexts, while also helping AI consumers select the right model for specific WordPress tasks
- The tool is written in Python, with dataset tests defined in JSON format for easy contribution
- Contribution is welcome, particularly in building out additional tests and datasets
@johnbillion and @raftaar1191 explored the operational aspects:
- @johnbillion asked whether WP Bench is running anywhere where results can be publicly viewed, or if it’s primarily a local-first tool; @shadyvb shared a test run for inspection
- @raftaar1191 offered to update the WP Bench documentation to clarify that it can be tested locally, particularly with Ollama, and to ensure users understand how to run it on their local machines
Observability, tracing, and evaluation
Several contributors raised questions about observability and evaluation methodologies:
- @justlevine asked whether there has been discussion about using WP Bench to evaluate changes to AGENTS.md or prompts in the AI Experiments plugin
- @jorgefilipecosta suggested that WP Bench could be used to evaluate AGENTS.md changes by running benchmarks and measuring score improvements
- @shadyvb inquired about existing plans for supporting observability and tracing with tools like LangSmith or LangFuse
The group acknowledged that datasets are currently very limited, and current benchmark scores should be taken with a grain of salt. There was general agreement on the value of moving beyond manual evaluation and anecdotal experience toward more systematic testing approaches.
@jason_the_adams noted that the PHP AI Client is experimenting with integration tests that run real prompts (https://github.com/WordPress/php-ai-client/pull/176), highlighting the difficulty of catching API edge cases through manual and unit testing alone. This work may have broader applications for the team’s testing strategy.
Given that integration tests require API keys, @jason_the_adams clarified that for now they are limiting execution to himself and @flixos90 running locally at their discretion before merging PRs or releasing, noting that there isn’t a precedent for WordPress storing API keys as GitHub secrets for this type of automation.
Next steps
- @flixos90 will explore and report back on the best approach to include PHP AI Client into core, similar to the Requests library pattern
- @isotropic will write the WP AI Client merge proposal for Make/Core
- @raftaar1191 will update WP Bench documentation to include guidance on local testing, particularly with Ollama
- The team will continue inviting contributors to test the WP AI Client
- Discussion on versionless Ability Deprecation should continue toward consensus on a solution
Upcoming meetings
Reminder that all team meetings are published to https://make.wordpress.org/meetings/#ai.
- The next bi-weekly AI Team Chat Slack discussion is scheduled for 5 February 2026.
- The next weekly AI Contributor Google Meet video call is scheduled for 28 January 2026.
Props to @jeffpaul for reviewing this post.