Briefly mentioning T205572: Optimize lang conversion and content negotiation combo - one of the decisions to make in the new design will be whether to cache un-language-converted content separately, as RESTBase does.

Editing team also was wondering about caching/updating linter data: T253799: RESTBase linting API is very slow (not cached).

I'm interested in bulk access for dumps purposes.

The ParserOutput object also extends a base class, CacheTime, which contains a bunch of ParserCache-specific expiry code. If this is appropriate for the new MPC implementation, we can include it in the base class we'd like to factor out of ParserOutput; if it is not, then we should keep it out of the base class of ParserOutput and include it (maybe as a trait) in the LegacyParserOutput used by the legacy parsercache and legacy parser.

kaldari moved this task from Untriaged to Later on the Platform Team Workboards (Platform-Product Roadmap) board.

@Naike, @kaldari, while we don't need this before EOQ3 and while this is strictly "Later", it might be useful thinking through the requirements sooner than later and see what kind of designs they induce ... in case we need to work and iterate through those.

Note that generation of e.g. HTML dumps (see T254275) in a way that does not require getting all content every time but only the changed items, may be facilitated by having revision id or timestamp tags available.

ParserCache needs to provide multi-bucket support and ability to tie them together with a key (revid / tid, etc.). Parsoid produces 3 components per page: HTML, data-parsoid JSON blob, and data-mw JSON blob. For networking and computational efficiency reasons, these are stored separately in RESTBase (minor detail: data-mw is not stored separately right now, but will be if RESTBase continues to be around). Not all Parsoid clients need all blobs. So, the API needs to be able to fetch individual blobs.

Currently RESTBase stores all blobs together in a JSON document { "html": "lalala", "data-parsoid": "trulala" }, fetches the whole thing during read and only returns the requested portion. Previously we indeed have stored parts of the page bundle separately in separate tables, but eventually simplified it with not visible performance impact. The performance considerations are tied to backend implementation (Cassandra in RESTBase case), so might not how true for MW ParserCache backend. However, I propose not to optimize prematurely and start with storing the whole page bundle. We can revision it later if we find the overhead of deserializing data-parsoid to be important.

ParserCache (or whatever code component it is) needs to support the stashing functionality for editing clients to provide "storage semantics" (instead of caching semantics where cached content can get evicted arbitrarily as far as clients are concerned) so presence of stashed content is guaranteed within session / time windows. RESTBase provides this.

I would like to separate these concerns, and have ParserCache concentrate on caching, and introduce a separate component for stashing at a later stage. The requirements for these two components are drastically different with ParserCache having 2-level cache deduplicating by used options, different expiry semantics and different key for access. ParserStash (name TBD) is a simple key-value with TTL expiry.

In addition to supporting RESTBase functionality, @EvanProdromou has framed this enhanced-ParserCache functionality as a Multi-Parser-Cache (MPC from here on) solution

This can probably be achieved in the beginning by introducing an entirely separate ParserCache, (ParsoidCache?) service, and using the appropriate one in appropriate places. Once we have tighter integration we can create a wrapper class that would route calls to the appropriate parser cache.

The ParserOutput object also extends a base class, CacheTime, which contains a bunch of ParserCache-specific expiry code. If this is appropriate for the new MPC implementation, we can include it in the base class we'd like to factor out of ParserOutput; if it is not, then we should keep it out of the base class of ParserOutput and include it (maybe as a trait) in the LegacyParserOutput used by the legacy parsercache and legacy parser.

I've been thinking to extract CacheTime interface (plus a few more methods) into CacheableParserOutput interface and make ParserCache work with any instance of CacheableParserOutput. Then we could either make Parsoid's PageBundle implement it, or create a wrapper implementing the interface. This is still not decided though, will keep the ticket updated with latest developments.

Some valuable info from @ssastry : https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/623032

Change 630382 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[mediawiki/core@master] Cover ParserCache with integration tests

https://gerrit.wikimedia.org/r/630382

Change 630382 merged by jenkins-bot:
[mediawiki/core@master] Cover ParserCache with integration tests

https://gerrit.wikimedia.org/r/630382

What about the stashing functionality that RESTBase provides? Or is that going to be handled as part of the "retire RESTBase" work?

In T250500#6721154, @ssastry wrote:

What about the stashing functionality that RESTBase provides? Or is that going to be handled as part of the "retire RESTBase" work?

T264669 is a new umbrella task for that work.

Good to know! Thanks! :)